Sep 09 13:24:21.860845 ip-10-0-136-35 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 13:24:21.860975 ip-10-0-136-35 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 13:24:21.861070 ip-10-0-136-35 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 13:24:21.861606 ip-10-0-136-35 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 13:24:31.947424 ip-10-0-136-35 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 13:24:31.947440 ip-10-0-136-35 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8d5949df84f743a8829ebc0ea756fc27 --
Sep 09 13:26:19.625709 ip-10-0-136-35 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 13:26:20.046264 ip-10-0-136-35 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 13:26:20.046264 ip-10-0-136-35 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 13:26:20.046264 ip-10-0-136-35 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 13:26:20.046264 ip-10-0-136-35 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 13:26:20.046264 ip-10-0-136-35 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 13:26:20.047279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.047193    2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 13:26:20.051556 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051537    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.051556 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051552    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.051556 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051560    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.051556 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051564    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051569    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051573    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051578    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051582    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051587    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051591    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051595    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051599    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051603    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051607    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051610    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051614    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051618    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051621    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051625    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051629    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051645    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051651    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.051807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051656    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051660    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051664    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051668    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051672    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051679    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051685    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051690    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051694    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051699    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051704    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051708    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051712    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051716    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051721    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051726    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051731    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051735    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051740    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051744    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.052445 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051748    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051752    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051757    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051761    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051765    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051769    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051774    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051778    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051782    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051786    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051790    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051795    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051800    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051803    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051807    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051811    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051815    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051821    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051825    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051830    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.052985 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051834    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051838    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051842    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051847    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051851    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051855    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051859    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051864    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051869    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051873    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051877    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051882    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051886    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051890    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051894    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051898    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051902    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051906    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051910    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051914    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.053807 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051918    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051922    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051926    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051930    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051937    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051943    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051948    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051953    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051957    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051962    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051966    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051971    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051975    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051979    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051984    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051988    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.054597 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.051992    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.055194 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055177    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.055194 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055193    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055199    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055204    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055207    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055210    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055213    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055216    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055220    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055224    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055227    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055231    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055234    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055236    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055239    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055242    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055245    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055248    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055250    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055253    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055256    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.055250 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055259    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055262    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055265    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055267    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055270    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055278    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055281    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055285    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055289    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055292    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055294    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055297    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055300    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055302    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055305    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055307    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055310    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055312    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055315    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.055774 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055317    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055320    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055322    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055324    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055327    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055329    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055332    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055334    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055337    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055339    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055342    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055345    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055348    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055351    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055353    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055357    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055360    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055363    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055365    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055368    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.056240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055370    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055372    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055376    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055378    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055381    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055384    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055386    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055390    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055392    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055395    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055398    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055400    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055403    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055406    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055408    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055411    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055414    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055416    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055419    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055421    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.056750 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055424    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055426    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055429    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055447    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055451    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055453    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055456    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055459    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055462    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055464    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055469    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055474    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055477    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055480    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055483    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055486    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055488    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055491    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055494    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.057247 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055857    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055862    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055865    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055868    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055871    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055874    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055876    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055879    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055882    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055886    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055890    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055893    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055896    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055899    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055902    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055904    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055908    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055911    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055913    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055916    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.057760 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055919    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055922    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055925    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055927    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055930    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055932    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055935    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055938    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055940    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055943    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055945    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055948    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055950    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055953    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055956    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055958    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055961    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055963    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055966    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055969    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.058249 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055971    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055974    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055976    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055980    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055983    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055987    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055989    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055992    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055995    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.055998    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056000    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056003    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056006    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056008    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056011    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056014    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056016    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056018    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056021    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.058764 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056023    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056027    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056031    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056034    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056037    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056040    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056042    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056045    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056048    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056051    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056054    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056056    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056059    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056062    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056064    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056067    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056069    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056072    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056074    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.059275 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056077    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056079    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056082    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056085    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056089    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056091    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056094    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056096    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056100    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056103    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056105    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056107    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056110    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056113    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056115    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056118    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056120    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056123    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056125    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056128    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.059752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056130    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056200    2559 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056236    2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056243    2559 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056247    2559 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056251    2559 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056254    2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056258    2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056263    2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056266    2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056269    2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056272    2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056275    2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056279    2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056282    2559 flags.go:64] FLAG: --cgroup-root=""
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056285    2559 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056288    2559 flags.go:64] FLAG: --client-ca-file=""
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056292    2559 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056295    2559 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056299    2559 flags.go:64] FLAG: --cluster-domain=""
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056302    2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056305    2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056308    2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056311    2559 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 13:26:20.060242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056315    2559 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056318    2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056321    2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056324    2559 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056327    2559 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056331    2559 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056334    2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056337    2559 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056340    2559 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056348    2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056351    2559 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056354    2559 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056357    2559 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056360    2559 flags.go:64] FLAG: --enable-server="true"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056362    2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056373    2559 flags.go:64] FLAG: --event-burst="100"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056376    2559 flags.go:64] FLAG: --event-qps="50"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056380    2559 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056383    2559 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056386    2559 flags.go:64] FLAG: --eviction-hard=""
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056394    2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056397    2559 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056399    2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056402    2559 flags.go:64] FLAG: --eviction-soft=""
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056406    2559 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 13:26:20.060870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056409    2559 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056412    2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056415    2559 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056418    2559 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056421    2559 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056424    2559 flags.go:64] FLAG: --feature-gates=""
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056427    2559 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056430    2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056447    2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056451    2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056454    2559 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056457    2559 flags.go:64] FLAG: --help="false"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056460    2559 flags.go:64] FLAG: --hostname-override="ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056463    2559 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056466    2559 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056469    2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056473    2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056476    2559 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056479    2559 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056482    2559 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056485    2559 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056488    2559 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056493    2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056496    2559 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 13:26:20.061501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056499    2559 flags.go:64] FLAG: --kube-reserved=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056502    2559 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056505    2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056508    2559 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056511    2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056514    2559 flags.go:64] FLAG: --lock-file=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056517    2559 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056520    2559 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056523    2559 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056528    2559 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056531    2559 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056534    2559 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056537    2559 flags.go:64] FLAG: --logging-format="text"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056540    2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056543    2559 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056546    2559 flags.go:64] FLAG: --manifest-url=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056549    2559 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056553    2559 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056557    2559 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056561    2559 flags.go:64] FLAG: --max-pods="110"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056563    2559 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056566    2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056569    2559 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056572    2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056575    2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 13:26:20.062104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056578    2559 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056581    2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056588    2559 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056591    2559 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056594    2559 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056598    2559 flags.go:64] FLAG: --pod-cidr=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056601    2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056606    2559 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056609    2559 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056612    2559 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056615    2559 flags.go:64] FLAG: --port="10250"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056618    2559 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056621    2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fdc93065ec9d7b6d"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056624    2559 flags.go:64] FLAG: --qos-reserved=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056627    2559 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056630    2559 flags.go:64] FLAG: --register-node="true"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056632    2559 flags.go:64] FLAG: --register-with-taints=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056636    2559 flags.go:64] FLAG: --registry-burst="10"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056639    2559 flags.go:64] FLAG: --registry-qps="5"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056642    2559 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056645    2559 flags.go:64] FLAG: --reserved-memory=""
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056649    2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056652    2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056655    2559 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056658    2559 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 13:26:20.062807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056660    2559 flags.go:64] FLAG: --runonce="false"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056663    2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056666    2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056669    2559 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056672    2559 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056675    2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056678    2559 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056681    2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056684    2559 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056687    2559 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056690    2559 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056692    2559 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056695    2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056700    2559 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056703    2559 flags.go:64] FLAG: --system-cgroups=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056706    2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056711    2559 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056714    2559 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056717    2559 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056721    2559 flags.go:64] FLAG: --tls-min-version=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056724    2559 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056727    2559 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056729    2559 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056732    2559 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056735    2559 flags.go:64] FLAG: --v="2"
Sep 09 13:26:20.063416 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056739    2559 flags.go:64] FLAG: --version="false"
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056743    2559 flags.go:64] FLAG: --vmodule=""
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056747    2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.056750    2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056873    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056877    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056881    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056884    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056887    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056891    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056893    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056896    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056899    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056902    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056905    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056909    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056913    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056915    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056920    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056923    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.064082 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056927    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056932    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056935    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056938    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056941    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056943    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056946    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056948    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056951    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056954    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056956    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056959    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056961    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056964    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056967    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056970    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056973    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056975    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056978    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.064693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056980    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056983    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056985    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056988    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056990    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056992    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056995    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.056997    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057000    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057003    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057005    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057008    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057010    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057014    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057018    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057021    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057023    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057026    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057028    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057032    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.065162 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057036    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057039    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057041    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057044    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057047    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057049    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057052    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057054    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057057    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057060    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057062    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057072    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057075    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057078    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057080    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057083    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057086    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057088    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057091    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057093    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.065664 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057095    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057098    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057100    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057103    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057106    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057110    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057114    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057117    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057119    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057122    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057125    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057128    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057131    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057134    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057136    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057139    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057141    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057144    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057147    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057149    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.066142 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057152    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057155    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057158    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.057160    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.057848    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.066515    2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.066529    2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.066542    2559 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.066548    2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066599    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066604    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066607    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066610    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066613    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066616    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.066640 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066619    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066622    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066625    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066628    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066631    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066634    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066637    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066640    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066643    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066647    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066650    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066653    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066656    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066658    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066661    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066663    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066666    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066668    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066671    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066674    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.067022 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066678    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066682    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066685    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066688    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066691    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066694    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066696    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066699    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066701    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066704    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066706    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066709    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066711    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066714    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066717    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066720    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066723    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066725    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066728    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066731    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.067558 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066733    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066736    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066738    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066741    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066743    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066746    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066749    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066751    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066755    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066759    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066762    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066764    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066767    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066769    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066772    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066774    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066777    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066780    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066782    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066785    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.068028 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066787    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066790    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066792    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066795    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066798    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066800    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066803    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066806    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066808    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066811    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066814    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066816    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066818    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066822    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066826    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066829    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066832    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066835    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066837    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.068577 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066840    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066843    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066846    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066848    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066851    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066853    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066856    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066858    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066861    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066863    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066866    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066869    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066871    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066873    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.069040 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.066878    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066981    2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066985    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066988    2559 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066991    2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066994    2559 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066997    2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.066999    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067001    2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067004    2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067007    2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067009    2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067012    2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067014    2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067017    2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067019    2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067022    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067024    2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067027    2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067030    2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 13:26:20.069384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067034    2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067038    2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067041    2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067043    2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067046    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067049    2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067051    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067054    2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067056    2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067060    2559 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067062    2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067065    2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067068    2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067070    2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067073    2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067075    2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067078    2559 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067080    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067083    2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067086    2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 13:26:20.069880 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067089    2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067092    2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067094    2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067096    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067099    2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067101    2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067104    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067106    2559 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067109    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067111    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067114    2559 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067116    2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067119    2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067121    2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067124    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067127    2559 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067130    2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067132    2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067135    2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067137    2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 13:26:20.070357 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067140    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067143    2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067146    2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067148    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067151    2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067153    2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067156    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067158    2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067162    2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067165    2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067169    2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067171    2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067174    2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067177    2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067180    2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067182    2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067185    2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067187    2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067190    2559 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 13:26:20.070867 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067193    2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067195    2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067197    2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067200    2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067202    2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067205    2559 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067207    2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067210    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067212    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067214    2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067218    2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067220    2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067223    2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067225    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067228    2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067230    2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067233    2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067236    2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067238    2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 13:26:20.071318 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067241    2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:20.067243    2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.067248    2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.067336    2559 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.069900    2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.070856    2559 server.go:1019] "Starting client certificate rotation"
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.070950    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 13:26:20.071792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.070981    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 13:26:20.073722 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.073708    2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 13:26:20.074740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.074722    2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 13:26:20.086354 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.086337    2559 log.go:25] "Validated CRI v1 runtime API"
Sep 09 13:26:20.090803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.090790    2559 log.go:25] "Validated CRI v1 image API"
Sep 09 13:26:20.091627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.091606    2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 13:26:20.096103 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.096083    2559 fs.go:135] Filesystem UUIDs: map[2e4cadab-5155-4ec2-9295-1a9efb5303b5:/dev/nvme0n1p4 5351fbb8-6b4f-4397-bb03-bf31dbe8c7ee:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Sep 09 13:26:20.096182 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.096101    2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 13:26:20.102380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.102266    2559 manager.go:216] Machine: {Timestamp:2026-09-09 13:26:20.099607743 +0000 UTC m=+0.362444393 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3097686 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e2fe3affc19ab4e1f1ccab3029e21 SystemUUID:ec2e2fe3-affc-19ab-4e1f-1ccab3029e21 BootID:8d5949df-84f7-43a8-829e-bc0ea756fc27 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5c:81:34:d4:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5c:81:34:d4:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:10:55:94:a5:e5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 13:26:20.102380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.102371    2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 13:26:20.102512 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.102484    2559 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 13:26:20.105496 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.105482    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 13:26:20.107059 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.107038    2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 13:26:20.107196 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.107061    2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 13:26:20.108004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.107994    2559 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 13:26:20.108038 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.108005    2559 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 13:26:20.108038 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.108013    2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 13:26:20.108038 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.108026    2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 13:26:20.109521 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.109512    2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 13:26:20.114250 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114230    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 13:26:20.114357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114347    2559 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 13:26:20.114391 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114385    2559 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 13:26:20.114428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114400    2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 13:26:20.114428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114413    2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 13:26:20.114428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114422    2559 kubelet.go:398] "Adding apiserver pod source"
Sep 09 13:26:20.114570 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114473    2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 13:26:20.114809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114796    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 13:26:20.114862 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.114819    2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 13:26:20.116950 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.116912    2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 13:26:20.120044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120022    2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 13:26:20.120112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120075    2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 13:26:20.120312 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120301    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 13:26:20.120352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120319    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 13:26:20.120352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120326    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 13:26:20.120352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120331    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 13:26:20.120352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120338    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 13:26:20.120352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.120344    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 13:26:20.121482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121472    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 13:26:20.121482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121482    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 13:26:20.121541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121492    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 13:26:20.121541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121498    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 13:26:20.121541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121506    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 13:26:20.121541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.121514    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 13:26:20.123185 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.123171    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 13:26:20.123185 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.123181    2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 13:26:20.126028 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.126001    2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 13:26:20.126028 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.126020    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 13:26:20.126122 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.126013    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 13:26:20.127017 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.127003    2559 server.go:1268] "Started kubelet"
Sep 09 13:26:20.127171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.127086    2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 13:26:20.127240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.127159    2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 13:26:20.127240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.127238    2559 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 13:26:20.128416 ip-10-0-136-35 systemd[1]: Started Kubernetes Kubelet.
Sep 09 13:26:20.128679 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.128658    2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 13:26:20.134175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.134158    2559 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 13:26:20.135341 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.135321    2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 13:26:20.137381 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.136166    2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-35.ec2.internal.18d3a9fcc6b765e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-35.ec2.internal,UID:ip-10-0-136-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-35.ec2.internal,},FirstTimestamp:2026-09-09 13:26:20.126979555 +0000 UTC m=+0.389816204,LastTimestamp:2026-09-09 13:26:20.126979555 +0000 UTC m=+0.389816204,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-35.ec2.internal,}"
Sep 09 13:26:20.137854 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.137837    2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 13:26:20.138544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.138529    2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 13:26:20.138735 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.138714    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.139292 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139278    2559 factory.go:55] Registering systemd factory
Sep 09 13:26:20.139367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139312    2559 factory.go:223] Registration of the systemd container factory successfully
Sep 09 13:26:20.139367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139337    2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 13:26:20.139367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139338    2559 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 13:26:20.139367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139357    2559 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 13:26:20.139544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139467    2559 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 13:26:20.139544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139476    2559 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 13:26:20.139544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139537    2559 factory.go:153] Registering CRI-O factory
Sep 09 13:26:20.139634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139548    2559 factory.go:223] Registration of the crio container factory successfully
Sep 09 13:26:20.139634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139589    2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 13:26:20.139634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139602    2559 factory.go:103] Registering Raw factory
Sep 09 13:26:20.139634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.139613    2559 manager.go:1196] Started watching for new ooms in manager
Sep 09 13:26:20.140046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.140033    2559 manager.go:349] Starting recovery of all containers
Sep 09 13:26:20.147150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.147118    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5xm55"
Sep 09 13:26:20.148573 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.148557    2559 manager.go:354] Recovery completed
Sep 09 13:26:20.152825 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.152808    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.154468 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.154444    2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 13:26:20.154629 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.154606    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.154696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.154639    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.154696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.154649    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.154786 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.154705    2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 13:26:20.155175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.155157    2559 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 13:26:20.155175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.155173    2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 13:26:20.155369 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.155189    2559 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 13:26:20.155369 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.155190    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.156637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.156624    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.156693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.156648    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.156693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.156658    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.157454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.157422    2559 policy_none.go:49] "None policy: Start"
Sep 09 13:26:20.157543 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.157474    2559 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 13:26:20.157543 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.157486    2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 13:26:20.159968 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.159952    2559 policy_none.go:47] "Start"
Sep 09 13:26:20.160052 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.160025    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5xm55"
Sep 09 13:26:20.185987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.185972    2559 manager.go:337] "Starting Device Plugin manager"
Sep 09 13:26:20.186063 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.186010    2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 13:26:20.186063 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186019    2559 server.go:87] "Starting device plugin registration server"
Sep 09 13:26:20.186312 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186299    2559 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 13:26:20.186356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186313    2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 13:26:20.186412 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186397    2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 13:26:20.186505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186492    2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 13:26:20.186505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.186505    2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 13:26:20.186954 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.186931    2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 13:26:20.187029 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.186988    2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.234775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.234746    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 13:26:20.237538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.235856    2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 13:26:20.237538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.235909    2559 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 13:26:20.237538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.235937    2559 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 13:26:20.237538 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.235979    2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 13:26:20.238474 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.238455    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 13:26:20.286539 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.286521    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.287658 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.287638    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.287746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.287670    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.287746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.287684    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.287746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.287707    2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.300022 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.299970    2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.300022 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.299989    2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-35.ec2.internal\": node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.336181 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.336156    2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"]
Sep 09 13:26:20.336271 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.336224    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.337015 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.336992    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.337083 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.337018    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.337083 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.337028    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.339455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.339428    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.339562 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.339548    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.339613 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.339577    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.340132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340117    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.340206 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340145    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.340253 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340236    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.340297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340252    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.340297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340261    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.340297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340269    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.340297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340276    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.340297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340282    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.340508 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.340235    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5067553cfb83a070b3ee7a935b901cd8-config\") pod \"kube-apiserver-proxy-ip-10-0-136-35.ec2.internal\" (UID: \"5067553cfb83a070b3ee7a935b901cd8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.342717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.342702    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.342796 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.342729    2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 13:26:20.343422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.343405    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 13:26:20.343511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.343448    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 13:26:20.343511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.343465    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeHasSufficientPID"
Sep 09 13:26:20.366913 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.366898    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-35.ec2.internal\" not found" node="ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.370467 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.370451    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.371301 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.371290    2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-35.ec2.internal\" not found" node="ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.440874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440851    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.440940 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440877    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.440940 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5067553cfb83a070b3ee7a935b901cd8-config\") pod \"kube-apiserver-proxy-ip-10-0-136-35.ec2.internal\" (UID: \"5067553cfb83a070b3ee7a935b901cd8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.440940 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440915    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.441043 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440937    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5067553cfb83a070b3ee7a935b901cd8-config\") pod \"kube-apiserver-proxy-ip-10-0-136-35.ec2.internal\" (UID: \"5067553cfb83a070b3ee7a935b901cd8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.441043 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.440951    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7eba1424af4a22818d143b17bdf0c05d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal\" (UID: \"7eba1424af4a22818d143b17bdf0c05d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.470684 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.470663    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.570742 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.570696    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.669709 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.669692    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.670762 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.670749    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.674465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:20.674151    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:20.771665 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.771638    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:20.872211 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:20.872124    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.346817 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:22.346793    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.445469 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.445443    2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 13:26:22.445872 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.445590    2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 13:26:22.447607 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:22.447590    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.537122 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.537081    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 13:21:21 +0000 UTC" deadline="2028-05-17 11:51:39.896506386 +0000 UTC"
Sep 09 13:26:22.537286 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.537115    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14782h25m17.359394518s"
Sep 09 13:26:22.549708 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:22.549533    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.632410 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:22.632381    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eba1424af4a22818d143b17bdf0c05d.slice/crio-2686ca64c2f6c6155b05918c78afe6e66d1cb3fc2cfdc570048e7b9db1ea71a1 WatchSource:0}: Error finding container 2686ca64c2f6c6155b05918c78afe6e66d1cb3fc2cfdc570048e7b9db1ea71a1: Status 404 returned error can't find the container with id 2686ca64c2f6c6155b05918c78afe6e66d1cb3fc2cfdc570048e7b9db1ea71a1
Sep 09 13:26:22.634196 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:22.634169    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5067553cfb83a070b3ee7a935b901cd8.slice/crio-e6037c760763dbee7b4de49983e1d67702221f4c7c1efd1cc32912ff80a58667 WatchSource:0}: Error finding container e6037c760763dbee7b4de49983e1d67702221f4c7c1efd1cc32912ff80a58667: Status 404 returned error can't find the container with id e6037c760763dbee7b4de49983e1d67702221f4c7c1efd1cc32912ff80a58667
Sep 09 13:26:22.636743 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.636725    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:26:22.650652 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:22.650633    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.654356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.654338    2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 13:26:22.715951 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.715930    2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 13:26:22.751464 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:22.751438    2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-35.ec2.internal\" not found"
Sep 09 13:26:22.834862 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.834798    2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 13:26:22.913212 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.913186    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:22.930516 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.930500    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 13:26:22.931594 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.931581    2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal"
Sep 09 13:26:22.943245 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:22.943227    2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 13:26:23.489379 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.489351    2559 apiserver.go:52] "Watching apiserver"
Sep 09 13:26:23.497360 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.497326    2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 13:26:23.498798 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.498316    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5","openshift-multus/multus-additional-cni-plugins-xrlmc","openshift-network-operator/iptables-alerter-bxvwp","openshift-ovn-kubernetes/ovnkube-node-9tqg4","kube-system/konnectivity-agent-xzxtz","openshift-cluster-node-tuning-operator/tuned-j674p","openshift-dns/node-resolver-v2xgb","openshift-image-registry/node-ca-6zfw7","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal","openshift-multus/multus-hk747","openshift-multus/network-metrics-daemon-nfsvq","openshift-network-diagnostics/network-check-target-p5cph","kube-system/global-pull-secret-syncer-gv79z"]
Sep 09 13:26:23.501350 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.501325    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.503318 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.503292    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:23.503441 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.503360    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:23.505611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.505454    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.505868 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.505844    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.505949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.505883    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t4f89\""
Sep 09 13:26:23.505949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.505849    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.507680 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.507662    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.510645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510555    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 13:26:23.510645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510601    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 13:26:23.510757 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510659    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-gnd48\""
Sep 09 13:26:23.510811 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510761    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.510811 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510802    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.510953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510846    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.510953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510870    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 13:26:23.510953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.510894    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.511126 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.511018    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x7bq4\""
Sep 09 13:26:23.511527 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.511510    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.511527 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.511519    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 13:26:23.513032 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.513013    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.513222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.513160    2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 13:26:23.514672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.514655    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 13:26:23.515276 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.515153    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.515276 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.515225    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:23.515734 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.515718    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.516109 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.516093    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.516801 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.516643    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 13:26:23.516801 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.516662    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 13:26:23.517671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517412    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 13:26:23.517671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517430    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.517671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517507    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.517671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517430    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fcclt\""
Sep 09 13:26:23.517922 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517738    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.517922 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.517798    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rg4zm\""
Sep 09 13:26:23.520591 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.520552    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.521047 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.521026    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 13:26:23.521971 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.521752    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h4qgs\""
Sep 09 13:26:23.522845 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.522825    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.523311 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-system-cni-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.523311 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523247    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2568ae2b-2def-4a07-bb76-298b5a88c049-iptables-alerter-script\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.523311 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523274    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-lib-modules\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523310    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-log-socket\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523342    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-script-lib\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523366    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523382    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpjfv\" (UniqueName: \"kubernetes.io/projected/cfb90b51-1423-4e1f-8771-82dd73ab0bce-kube-api-access-rpjfv\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-tmp\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523426    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-systemd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523450    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-ovn\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-node-log\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523484    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523500    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovn-node-metrics-cert\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523519    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94hsz\" (UniqueName: \"kubernetes.io/projected/ffa1b527-a17f-4123-9ca0-33cf66176215-kube-api-access-94hsz\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523542    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-etc-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-modprobe-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523621    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523678    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-tuned\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523711    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-netns\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-var-lib-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysconfig\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523783    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-slash\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523797    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523809    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fq2k\" (UniqueName: \"kubernetes.io/projected/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-kube-api-access-9fq2k\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523831    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffa1b527-a17f-4123-9ca0-33cf66176215-hosts-file\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523852    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa1b527-a17f-4123-9ca0-33cf66176215-tmp-dir\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523887    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trmxf\" (UniqueName: \"kubernetes.io/projected/36eed273-3006-4797-978e-bf5d1f555385-kube-api-access-trmxf\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.523977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523914    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523965    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-host\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.523989    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kxtg\" (UniqueName: \"kubernetes.io/projected/e7257a2c-c31d-4511-b712-4099f1fc0dec-kube-api-access-8kxtg\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524013    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-netd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524084    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2568ae2b-2def-4a07-bb76-298b5a88c049-host-slash\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524559    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-conf\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524618    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-systemd\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524618    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524644    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2nspd\""
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524660    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-bin\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524696    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-config\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.524717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524719    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524720    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-env-overrides\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g27sw\" (UniqueName: \"kubernetes.io/projected/2568ae2b-2def-4a07-bb76-298b5a88c049-kube-api-access-g27sw\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524838    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-run\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-var-lib-kubelet\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524880    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-systemd-units\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524897    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cnibin\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524919    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524942    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524967    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-kubernetes\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524987    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-sys\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.524994    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.525007    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-kubelet\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.525028    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.525258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.525079    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 13:26:23.525966 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.525302    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 13:26:23.525966 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.525469    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4bbxn\""
Sep 09 13:26:23.526100 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.526082    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.528496 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.528436    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.528496 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.528490    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:23.528988 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.528784    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 13:26:23.528988 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.528880    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-5qtgs\""
Sep 09 13:26:23.529566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.529526    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 13:26:23.531936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.531825    2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 13:26:23.566035 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.566014    2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7pwk"
Sep 09 13:26:23.575272 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.575249    2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f7pwk"
Sep 09 13:26:23.614400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.614337    2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 13:26:23.617396 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.617355    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal" event={"ID":"5067553cfb83a070b3ee7a935b901cd8","Type":"ContainerStarted","Data":"e6037c760763dbee7b4de49983e1d67702221f4c7c1efd1cc32912ff80a58667"}
Sep 09 13:26:23.618541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.618518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal" event={"ID":"7eba1424af4a22818d143b17bdf0c05d","Type":"ContainerStarted","Data":"2686ca64c2f6c6155b05918c78afe6e66d1cb3fc2cfdc570048e7b9db1ea71a1"}
Sep 09 13:26:23.625673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-node-log\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.625763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovn-node-metrics-cert\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.625763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-hostroot\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.625763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625719    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-multus-certs\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.625763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625742    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-registration-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.625763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625756    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-dbus\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-var-lib-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625787    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trmxf\" (UniqueName: \"kubernetes.io/projected/36eed273-3006-4797-978e-bf5d1f555385-kube-api-access-trmxf\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625808    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-slash\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625830    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fq2k\" (UniqueName: \"kubernetes.io/projected/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-kube-api-access-9fq2k\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625851    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-agent-certs\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625870    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-host\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625884    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-conf-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625910    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-device-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.625935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625938    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-conf\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625953    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-bin\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625973    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-env-overrides\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.625993    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626008    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-run\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626022    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-var-lib-kubelet\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626035    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-systemd-units\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626057    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-socket-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626080    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zv8d4\" (UniqueName: \"kubernetes.io/projected/338cedfc-2d55-4430-9472-42e3415b4ba1-kube-api-access-zv8d4\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626190    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-run\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-var-lib-kubelet\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626257    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-systemd-units\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626257    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.626325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626285    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-node-log\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-host\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626338    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-var-lib-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626438    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-slash\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626526    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-bin\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626561    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6x2w\" (UniqueName: \"kubernetes.io/projected/c75d4892-66db-4119-bd92-5b343db8b44e-kube-api-access-k6x2w\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-conf\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626595    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-env-overrides\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626633    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-system-cni-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626648    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2568ae2b-2def-4a07-bb76-298b5a88c049-iptables-alerter-script\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626669    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-log-socket\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626704    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-socket-dir-parent\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.626708    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626720    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-kubelet\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-systemd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.626934 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.626776    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:24.12675391 +0000 UTC m=+3.015453883 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626795    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-systemd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626810    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626837    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-log-socket\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626841    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94hsz\" (UniqueName: \"kubernetes.io/projected/ffa1b527-a17f-4123-9ca0-33cf66176215-kube-api-access-94hsz\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626866    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-etc-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-modprobe-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626913    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626940    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-tuned\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626975    2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626990    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-netns\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627013    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffa1b527-a17f-4123-9ca0-33cf66176215-hosts-file\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-modprobe-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa1b527-a17f-4123-9ca0-33cf66176215-tmp-dir\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysconfig\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-etc-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.627775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627135    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627154    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2568ae2b-2def-4a07-bb76-298b5a88c049-host-slash\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627182    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2568ae2b-2def-4a07-bb76-298b5a88c049-iptables-alerter-script\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627213    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2568ae2b-2def-4a07-bb76-298b5a88c049-host-slash\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysconfig\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627298    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-sysctl-d\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.626892    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-system-cni-dir\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-ovn-kubernetes\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627366    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-run-netns\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g27sw\" (UniqueName: \"kubernetes.io/projected/2568ae2b-2def-4a07-bb76-298b5a88c049-kube-api-access-g27sw\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627410    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffa1b527-a17f-4123-9ca0-33cf66176215-tmp-dir\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627418    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffa1b527-a17f-4123-9ca0-33cf66176215-hosts-file\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627427    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-system-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627451    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-cni-binary-copy\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627474    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-etc-selinux\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.628647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kxtg\" (UniqueName: \"kubernetes.io/projected/e7257a2c-c31d-4511-b712-4099f1fc0dec-kube-api-access-8kxtg\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627524    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-netd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627545    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-k8s-cni-cncf-io\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627567    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945gf\" (UniqueName: \"kubernetes.io/projected/65bd199c-778e-45f1-a136-bc1ef27587ea-kube-api-access-945gf\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627606    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627626    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-systemd\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627640    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-config\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627654    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-cnibin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627668    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-etc-kubernetes\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627672    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-cni-netd\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627681    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-sys-fs\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627695    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c75d4892-66db-4119-bd92-5b343db8b44e-serviceca\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cnibin\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627722    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-systemd\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627765    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cni-binary-copy\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627830    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-netns\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627833    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cfb90b51-1423-4e1f-8771-82dd73ab0bce-cnibin\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627882    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c75d4892-66db-4119-bd92-5b343db8b44e-host\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627912    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-konnectivity-ca\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627947    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627984    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-openvswitch\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.627987    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-kubernetes\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-sys\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-kubernetes\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628068    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-sys\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628090    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-kubelet\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/cfb90b51-1423-4e1f-8771-82dd73ab0bce-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628152    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-host-kubelet\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628133    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-bin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628189    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628226    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-lib-modules\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628250    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-config\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.629999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628269    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-script-lib\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628300    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpjfv\" (UniqueName: \"kubernetes.io/projected/cfb90b51-1423-4e1f-8771-82dd73ab0bce-kube-api-access-rpjfv\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628311    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e7257a2c-c31d-4511-b712-4099f1fc0dec-lib-modules\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628327    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-multus\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628373    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-daemon-config\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-kubelet-config\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628425    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-tmp\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-ovn\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628525    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-run-ovn\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.630637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.628729    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovnkube-script-lib\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.631128 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.631111    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-tmp\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.631231 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.631213    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-ovn-node-metrics-cert\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.631607 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.631586    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e7257a2c-c31d-4511-b712-4099f1fc0dec-etc-tuned\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.653216 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.652948    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:23.653216 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.652974    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:23.653216 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.652992    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:23.653216 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.653047    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:24.153029465 +0000 UTC m=+3.041729451 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.654846    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpjfv\" (UniqueName: \"kubernetes.io/projected/cfb90b51-1423-4e1f-8771-82dd73ab0bce-kube-api-access-rpjfv\") pod \"multus-additional-cni-plugins-xrlmc\" (UID: \"cfb90b51-1423-4e1f-8771-82dd73ab0bce\") " pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.655785    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kxtg\" (UniqueName: \"kubernetes.io/projected/e7257a2c-c31d-4511-b712-4099f1fc0dec-kube-api-access-8kxtg\") pod \"tuned-j674p\" (UID: \"e7257a2c-c31d-4511-b712-4099f1fc0dec\") " pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.656437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fq2k\" (UniqueName: \"kubernetes.io/projected/8d2b4486-fdea-4c96-bdd2-cb0232ff47ce-kube-api-access-9fq2k\") pod \"ovnkube-node-9tqg4\" (UID: \"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce\") " pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.656530    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g27sw\" (UniqueName: \"kubernetes.io/projected/2568ae2b-2def-4a07-bb76-298b5a88c049-kube-api-access-g27sw\") pod \"iptables-alerter-bxvwp\" (UID: \"2568ae2b-2def-4a07-bb76-298b5a88c049\") " pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.656836    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94hsz\" (UniqueName: \"kubernetes.io/projected/ffa1b527-a17f-4123-9ca0-33cf66176215-kube-api-access-94hsz\") pod \"node-resolver-v2xgb\" (UID: \"ffa1b527-a17f-4123-9ca0-33cf66176215\") " pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.658555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.657467    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trmxf\" (UniqueName: \"kubernetes.io/projected/36eed273-3006-4797-978e-bf5d1f555385-kube-api-access-trmxf\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:23.728707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-cnibin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728711    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-etc-kubernetes\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728726    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-sys-fs\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c75d4892-66db-4119-bd92-5b343db8b44e-serviceca\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728771    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728794    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-netns\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728799    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-cnibin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c75d4892-66db-4119-bd92-5b343db8b44e-host\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-sys-fs\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-konnectivity-ca\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728851    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-netns\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.728875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728811    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-etc-kubernetes\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728885    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-bin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c75d4892-66db-4119-bd92-5b343db8b44e-host\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728911    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728942    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-multus\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-daemon-config\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728983    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-bin\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.728996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-kubelet-config\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.729018    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-hostroot\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729027    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-cni-multus\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-multus-certs\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729066    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-hostroot\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:23.729081    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:24.22906287 +0000 UTC m=+3.117762840 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729087    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-multus-certs\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-registration-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729149    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-dbus\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729178    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-agent-certs\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729185    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c75d4892-66db-4119-bd92-5b343db8b44e-serviceca\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729200    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-conf-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-device-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729248    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-kubelet-config\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-socket-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729267    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-dbus\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729284    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zv8d4\" (UniqueName: \"kubernetes.io/projected/338cedfc-2d55-4430-9472-42e3415b4ba1-kube-api-access-zv8d4\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-conf-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6x2w\" (UniqueName: \"kubernetes.io/projected/c75d4892-66db-4119-bd92-5b343db8b44e-kube-api-access-k6x2w\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729314    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-registration-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729362    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-device-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729362    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-socket-dir-parent\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729376    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-konnectivity-ca\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729398    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-kubelet\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729402    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-socket-dir-parent\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.729944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729447    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-system-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-cni-binary-copy\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729493    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-socket-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729495    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-etc-selinux\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729535    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-k8s-cni-cncf-io\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729561    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-945gf\" (UniqueName: \"kubernetes.io/projected/65bd199c-778e-45f1-a136-bc1ef27587ea-kube-api-access-945gf\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729600    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-multus-daemon-config\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729602    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729628    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-var-lib-kubelet\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729633    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-host-run-k8s-cni-cncf-io\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729644    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/65bd199c-778e-45f1-a136-bc1ef27587ea-system-cni-dir\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-etc-selinux\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.729682    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/338cedfc-2d55-4430-9472-42e3415b4ba1-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.730662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.730055    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/65bd199c-778e-45f1-a136-bc1ef27587ea-cni-binary-copy\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.732146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.732126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/912c2218-a6f4-40ff-b5e4-d7e06ce737e2-agent-certs\") pod \"konnectivity-agent-xzxtz\" (UID: \"912c2218-a6f4-40ff-b5e4-d7e06ce737e2\") " pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.741139 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.741084    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zv8d4\" (UniqueName: \"kubernetes.io/projected/338cedfc-2d55-4430-9472-42e3415b4ba1-kube-api-access-zv8d4\") pod \"aws-ebs-csi-driver-node-wx9b5\" (UID: \"338cedfc-2d55-4430-9472-42e3415b4ba1\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.741923 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.741885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-945gf\" (UniqueName: \"kubernetes.io/projected/65bd199c-778e-45f1-a136-bc1ef27587ea-kube-api-access-945gf\") pod \"multus-hk747\" (UID: \"65bd199c-778e-45f1-a136-bc1ef27587ea\") " pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.742395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.742376    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6x2w\" (UniqueName: \"kubernetes.io/projected/c75d4892-66db-4119-bd92-5b343db8b44e-kube-api-access-k6x2w\") pod \"node-ca-6zfw7\" (UID: \"c75d4892-66db-4119-bd92-5b343db8b44e\") " pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.814789 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.814755    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j674p"
Sep 09 13:26:23.824737 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.824712    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xrlmc"
Sep 09 13:26:23.834096 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.834075    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bxvwp"
Sep 09 13:26:23.843642 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.843622    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:23.851224 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.851195    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-v2xgb"
Sep 09 13:26:23.858790 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.858774    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-hk747"
Sep 09 13:26:23.866231 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.866213    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5"
Sep 09 13:26:23.872183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.872165    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6zfw7"
Sep 09 13:26:23.878726 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.878706    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:23.990232 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:23.990207    2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 13:26:24.132295 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.132220    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:24.132425 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.132360    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:24.132425 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.132419    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:25.132403082 +0000 UTC m=+4.021103056 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:24.232559 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.232527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:24.232715 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.232580    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:24.232715 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232671    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:24.232715 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232691    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:24.232715 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232713    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:24.232835 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232720    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:25.232708191 +0000 UTC m=+4.121408159 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:24.232835 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232722    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:24.232835 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:24.232763    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:25.232752175 +0000 UTC m=+4.121452148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:24.537658 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.537634    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfb90b51_1423_4e1f_8771_82dd73ab0bce.slice/crio-1fd4f79a14d24a7a5b38df8ec6b01587df1b23b18ea219626156ac0b731b072e WatchSource:0}: Error finding container 1fd4f79a14d24a7a5b38df8ec6b01587df1b23b18ea219626156ac0b731b072e: Status 404 returned error can't find the container with id 1fd4f79a14d24a7a5b38df8ec6b01587df1b23b18ea219626156ac0b731b072e
Sep 09 13:26:24.538759 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.538728    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912c2218_a6f4_40ff_b5e4_d7e06ce737e2.slice/crio-7a09647a8b08734e23230d5e38e20d626e60db6c21bf30f8a38406f3a30a5be9 WatchSource:0}: Error finding container 7a09647a8b08734e23230d5e38e20d626e60db6c21bf30f8a38406f3a30a5be9: Status 404 returned error can't find the container with id 7a09647a8b08734e23230d5e38e20d626e60db6c21bf30f8a38406f3a30a5be9
Sep 09 13:26:24.540212 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.540188    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffa1b527_a17f_4123_9ca0_33cf66176215.slice/crio-0567257f6b40d23d712911308cd69a9c6599c762aacafabdb176620007793a51 WatchSource:0}: Error finding container 0567257f6b40d23d712911308cd69a9c6599c762aacafabdb176620007793a51: Status 404 returned error can't find the container with id 0567257f6b40d23d712911308cd69a9c6599c762aacafabdb176620007793a51
Sep 09 13:26:24.543086 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.543060    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75d4892_66db_4119_bd92_5b343db8b44e.slice/crio-7a07a55bb09040ff8b7975e4f26747fdc7b0e2e78c26c5ee303a0a6a8727e0d4 WatchSource:0}: Error finding container 7a07a55bb09040ff8b7975e4f26747fdc7b0e2e78c26c5ee303a0a6a8727e0d4: Status 404 returned error can't find the container with id 7a07a55bb09040ff8b7975e4f26747fdc7b0e2e78c26c5ee303a0a6a8727e0d4
Sep 09 13:26:24.543527 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.543506    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2568ae2b_2def_4a07_bb76_298b5a88c049.slice/crio-f3f8509f3ed530b228772e8e900ed15735e6caa11938f2d0a3e4f6938f91663e WatchSource:0}: Error finding container f3f8509f3ed530b228772e8e900ed15735e6caa11938f2d0a3e4f6938f91663e: Status 404 returned error can't find the container with id f3f8509f3ed530b228772e8e900ed15735e6caa11938f2d0a3e4f6938f91663e
Sep 09 13:26:24.544420 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.544398    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338cedfc_2d55_4430_9472_42e3415b4ba1.slice/crio-49b0c542ad9c484a98762d20fcf6fb2e3ba740f10f69d01224da35ef5ec28525 WatchSource:0}: Error finding container 49b0c542ad9c484a98762d20fcf6fb2e3ba740f10f69d01224da35ef5ec28525: Status 404 returned error can't find the container with id 49b0c542ad9c484a98762d20fcf6fb2e3ba740f10f69d01224da35ef5ec28525
Sep 09 13:26:24.546021 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.545941    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2b4486_fdea_4c96_bdd2_cb0232ff47ce.slice/crio-6004eb90978cdba0df3211cb4a68c155f7ac87d4fc435324e74286064c68aa68 WatchSource:0}: Error finding container 6004eb90978cdba0df3211cb4a68c155f7ac87d4fc435324e74286064c68aa68: Status 404 returned error can't find the container with id 6004eb90978cdba0df3211cb4a68c155f7ac87d4fc435324e74286064c68aa68
Sep 09 13:26:24.546190 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.546168    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7257a2c_c31d_4511_b712_4099f1fc0dec.slice/crio-db98c972e7c0c6a666b42b521d812a68c33b8eabbdb97c2656c28245f6937ff1 WatchSource:0}: Error finding container db98c972e7c0c6a666b42b521d812a68c33b8eabbdb97c2656c28245f6937ff1: Status 404 returned error can't find the container with id db98c972e7c0c6a666b42b521d812a68c33b8eabbdb97c2656c28245f6937ff1
Sep 09 13:26:24.548408 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:24.548388    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65bd199c_778e_45f1_a136_bc1ef27587ea.slice/crio-dab79bfa7a2d11931abd3b3573d4ced3f079b9e47a0b3bf9ff30ad1be984856e WatchSource:0}: Error finding container dab79bfa7a2d11931abd3b3573d4ced3f079b9e47a0b3bf9ff30ad1be984856e: Status 404 returned error can't find the container with id dab79bfa7a2d11931abd3b3573d4ced3f079b9e47a0b3bf9ff30ad1be984856e
Sep 09 13:26:24.578259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.578235    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 13:21:23 +0000 UTC" deadline="2028-06-07 19:04:49.573920962 +0000 UTC"
Sep 09 13:26:24.578259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.578258    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15293h38m24.995665552s"
Sep 09 13:26:24.623206 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.623174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk747" event={"ID":"65bd199c-778e-45f1-a136-bc1ef27587ea","Type":"ContainerStarted","Data":"dab79bfa7a2d11931abd3b3573d4ced3f079b9e47a0b3bf9ff30ad1be984856e"}
Sep 09 13:26:24.626287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.626229    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6zfw7" event={"ID":"c75d4892-66db-4119-bd92-5b343db8b44e","Type":"ContainerStarted","Data":"7a07a55bb09040ff8b7975e4f26747fdc7b0e2e78c26c5ee303a0a6a8727e0d4"}
Sep 09 13:26:24.628173 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.628132    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2xgb" event={"ID":"ffa1b527-a17f-4123-9ca0-33cf66176215","Type":"ContainerStarted","Data":"0567257f6b40d23d712911308cd69a9c6599c762aacafabdb176620007793a51"}
Sep 09 13:26:24.629405 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.629382    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xzxtz" event={"ID":"912c2218-a6f4-40ff-b5e4-d7e06ce737e2","Type":"ContainerStarted","Data":"7a09647a8b08734e23230d5e38e20d626e60db6c21bf30f8a38406f3a30a5be9"}
Sep 09 13:26:24.630541 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.630520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerStarted","Data":"1fd4f79a14d24a7a5b38df8ec6b01587df1b23b18ea219626156ac0b731b072e"}
Sep 09 13:26:24.631964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.631939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j674p" event={"ID":"e7257a2c-c31d-4511-b712-4099f1fc0dec","Type":"ContainerStarted","Data":"db98c972e7c0c6a666b42b521d812a68c33b8eabbdb97c2656c28245f6937ff1"}
Sep 09 13:26:24.633172 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.633146    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"6004eb90978cdba0df3211cb4a68c155f7ac87d4fc435324e74286064c68aa68"}
Sep 09 13:26:24.634527 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.634490    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5" event={"ID":"338cedfc-2d55-4430-9472-42e3415b4ba1","Type":"ContainerStarted","Data":"49b0c542ad9c484a98762d20fcf6fb2e3ba740f10f69d01224da35ef5ec28525"}
Sep 09 13:26:24.635675 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:24.635650    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bxvwp" event={"ID":"2568ae2b-2def-4a07-bb76-298b5a88c049","Type":"ContainerStarted","Data":"f3f8509f3ed530b228772e8e900ed15735e6caa11938f2d0a3e4f6938f91663e"}
Sep 09 13:26:25.137759 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.137682    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:25.137915 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.137829    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:25.137915 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.137886    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:27.137867106 +0000 UTC m=+6.026567094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.238740    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.238833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.238959    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.239015    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:27.238998333 +0000 UTC m=+6.127698309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.239421    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.239438    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.239450    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:25.239530 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.239494    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:27.239479911 +0000 UTC m=+6.128179885 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:25.578703 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.578562    2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 13:21:23 +0000 UTC" deadline="2028-06-04 11:27:38.240027998 +0000 UTC"
Sep 09 13:26:25.578703 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.578628    2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15214h1m12.661403851s"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.615678    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.615791    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.616163    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.616255    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.616317    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:25.616425 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:25.616387    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:25.643051 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.643024    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal" event={"ID":"5067553cfb83a070b3ee7a935b901cd8","Type":"ContainerStarted","Data":"ec21a97a5bf0394060016ee637dccdb7919b6a6790eb43cf62660f18f898b554"}
Sep 09 13:26:25.648289 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.648265    2559 generic.go:356] "Generic (PLEG): container finished" podID="7eba1424af4a22818d143b17bdf0c05d" containerID="c55f2bcfb91540eda9e5aa9a4fc029599fa3e6d7e11f643006ee9269481688d6" exitCode=0
Sep 09 13:26:25.648446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.648307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal" event={"ID":"7eba1424af4a22818d143b17bdf0c05d","Type":"ContainerDied","Data":"c55f2bcfb91540eda9e5aa9a4fc029599fa3e6d7e11f643006ee9269481688d6"}
Sep 09 13:26:25.694320 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:25.694265    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-35.ec2.internal" podStartSLOduration=3.694245638 podStartE2EDuration="3.694245638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:25.661514169 +0000 UTC m=+4.550214160" watchObservedRunningTime="2026-09-09 13:26:25.694245638 +0000 UTC m=+4.582945609"
Sep 09 13:26:26.655949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:26.655623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal" event={"ID":"7eba1424af4a22818d143b17bdf0c05d","Type":"ContainerStarted","Data":"015b724a0d52d5f986a1494a268c117054bca667cb4c5a8a0ddae19f4ad86d7b"}
Sep 09 13:26:26.673652 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:26.673594    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-35.ec2.internal" podStartSLOduration=4.673555403 podStartE2EDuration="4.673555403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:26.673539135 +0000 UTC m=+5.562239125" watchObservedRunningTime="2026-09-09 13:26:26.673555403 +0000 UTC m=+5.562255394"
Sep 09 13:26:27.154786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.154702    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:27.154940 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.154858    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:27.154940 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.154931    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:31.15491254 +0000 UTC m=+10.043612519 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:27.255640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.255599    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:27.255806 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.255676    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:27.255882 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255811    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:27.255882 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255830    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:27.255882 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255847    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:27.255882 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255859    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:27.256076 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255886    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:31.255858556 +0000 UTC m=+10.144558542 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:27.256076 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.255903    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:31.255895348 +0000 UTC m=+10.144595319 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:27.611087 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.611004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:27.611087 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.611037    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:27.611304 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:27.611004    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:27.611304 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.611144    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:27.611304 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.611191    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:27.611304 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:27.611266    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:29.611397 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:29.611296    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:29.611858 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:29.611443    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:29.611858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:29.611812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:29.611963 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:29.611908    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:29.611963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:29.611952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:29.612088 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:29.612053    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:31.190962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.190928    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:31.191366 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.191056    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:31.191366 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.191108    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:39.191094321 +0000 UTC m=+18.079794289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:31.292174 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.292074    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:31.292174 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.292155    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292212    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292276    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292298    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292279    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:39.292259954 +0000 UTC m=+18.180959927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292310    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:31.292391 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.292363    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:39.292347343 +0000 UTC m=+18.181047329 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.611474    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.611603    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.611957    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.612040    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:31.612102    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:31.612259 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:31.612181    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:33.610855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:33.610745    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:33.610855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:33.610768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:33.611335 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:33.610866    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:33.611335 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:33.610919    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:33.611335 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:33.611006    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:33.611335 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:33.611071    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:35.610907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:35.610868    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:35.611325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:35.610875    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:35.611325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:35.610999    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:35.611325 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:35.610994    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:35.611325 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:35.611088    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:35.611325 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:35.611170    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:37.610605 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:37.610558    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:37.611072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:37.610559    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:37.611072 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:37.610701    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:37.611072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:37.610563    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:37.611072 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:37.610793    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:37.611072 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:37.610879    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:39.248129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.248089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:39.248605 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.248264    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:39.248605 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.248341    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:55.248320064 +0000 UTC m=+34.137020049 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:39.348725 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.348689    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:39.348901 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.348743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:39.348901 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.348858    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:39.349010 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.348917    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:55.348898454 +0000 UTC m=+34.237598424 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:39.349010 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.348860    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:39.349010 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.348972    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:39.349010 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.348987    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:39.349216 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.349042    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:55.349021076 +0000 UTC m=+34.237721050 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:39.610856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.610776    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:39.611014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.610780    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:39.611014 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.610894    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:39.611014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:39.610785    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:39.611130 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.611012    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:39.612476 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:39.612429    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:41.611535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:41.611502    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:41.611997 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:41.611624    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:41.611997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:41.611648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:41.611997 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:41.611728    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:41.611997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:41.611648    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:41.611997 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:41.611818    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:42.688707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.688438    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j674p" event={"ID":"e7257a2c-c31d-4511-b712-4099f1fc0dec","Type":"ContainerStarted","Data":"92d315f07d25987b72da5fb20a26fbb26e6ac3edc93e144042d0f5a39c6d99d5"}
Sep 09 13:26:42.691210 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691175    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"79ab4a1450b77f928291bef5e4c3a803c22cee3a4b528adb4153d5a667f1bb55"}
Sep 09 13:26:42.691326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"5b425202958ff70f2cffbc255a63b990f7f0798251b7cf844b3f8ef1b69550d7"}
Sep 09 13:26:42.691326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"d9a988a595bf030040cbf8757badc2f377be2e3dc8d46642dd68a93bed70490b"}
Sep 09 13:26:42.691326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691309    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"064a65646723eb18bc383ef604b247acc285f982ed4a46edd271f9005fea11a9"}
Sep 09 13:26:42.691443 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"7e60e693e38200f9a6acde8c735a9aa7f77a2006a7ef19678e26346f6128edb5"}
Sep 09 13:26:42.691443 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.691372    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"6b86ffa4f2bdae299413b291b96bd82c8355188493380fa12d423b5ed704238c"}
Sep 09 13:26:42.692545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.692523    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5" event={"ID":"338cedfc-2d55-4430-9472-42e3415b4ba1","Type":"ContainerStarted","Data":"d360e57cbb6c01b15a833f896816e9405bc98063b62ebf61653ad7cbc5287627"}
Sep 09 13:26:42.693784 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.693759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-hk747" event={"ID":"65bd199c-778e-45f1-a136-bc1ef27587ea","Type":"ContainerStarted","Data":"6b0a9ab87c5b6967466f770c0528ff4de5b700031128e7a9bd3fc85e7347d364"}
Sep 09 13:26:42.695076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.695055    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6zfw7" event={"ID":"c75d4892-66db-4119-bd92-5b343db8b44e","Type":"ContainerStarted","Data":"6e59e0ac1003807d209439fa3246996945a947ed4ad91760f560135ba9c7fa7f"}
Sep 09 13:26:42.696315 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.696297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-v2xgb" event={"ID":"ffa1b527-a17f-4123-9ca0-33cf66176215","Type":"ContainerStarted","Data":"46fb1d86dcf7f01e0c2a3513ffa68f90525a54680cad8caf97fa788e345253f1"}
Sep 09 13:26:42.697520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.697501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xzxtz" event={"ID":"912c2218-a6f4-40ff-b5e4-d7e06ce737e2","Type":"ContainerStarted","Data":"dd7abb907660baff7250f575ff294fa125c2a4e39cbc00cdbb5fda29a9eebc1e"}
Sep 09 13:26:42.698797 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.698779    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="127961dfa1385d477f9bc187747d229fd0d52cbe595055d3c2b0b328afb6835c" exitCode=0
Sep 09 13:26:42.698898 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.698804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"127961dfa1385d477f9bc187747d229fd0d52cbe595055d3c2b0b328afb6835c"}
Sep 09 13:26:42.755272 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.755234    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j674p" podStartSLOduration=4.424028336 podStartE2EDuration="21.755220927s" totalImagesPullingTime="17.331192591s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:42.721552221 +0000 UTC m=+21.610252211" watchObservedRunningTime="2026-09-09 13:26:42.755220927 +0000 UTC m=+21.643920915"
Sep 09 13:26:42.788852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.788822    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xzxtz" podStartSLOduration=3.437226305 podStartE2EDuration="20.788810721s" totalImagesPullingTime="17.351584416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:42.786082784 +0000 UTC m=+21.674782774" watchObservedRunningTime="2026-09-09 13:26:42.788810721 +0000 UTC m=+21.677510710"
Sep 09 13:26:42.788965 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.788946    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6zfw7" podStartSLOduration=3.916210663 podStartE2EDuration="20.788941629s" totalImagesPullingTime="16.872730966s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:42.758608222 +0000 UTC m=+21.647308212" watchObservedRunningTime="2026-09-09 13:26:42.788941629 +0000 UTC m=+21.677641681"
Sep 09 13:26:42.867124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:42.867082    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-hk747" podStartSLOduration=4.53018403 podStartE2EDuration="21.867065581s" totalImagesPullingTime="17.336881551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:42.866608389 +0000 UTC m=+21.755308380" watchObservedRunningTime="2026-09-09 13:26:42.867065581 +0000 UTC m=+21.755765572"
Sep 09 13:26:43.356745 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.356724    2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 13:26:43.575556 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.575429    2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T13:26:43.35674179Z","UUID":"26af546f-5448-4ee8-b6f2-ac275d8d12c9","Handler":null,"Name":"","Endpoint":""}
Sep 09 13:26:43.576951 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.576931    2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 13:26:43.576951 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.576964    2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 13:26:43.611016 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.610995    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:43.611129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.611100    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:43.611129 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:43.611114    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:43.611259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.611133    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:43.611259 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:43.611214    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:43.611345 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:43.611300    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:43.703082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.703049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5" event={"ID":"338cedfc-2d55-4430-9472-42e3415b4ba1","Type":"ContainerStarted","Data":"3920b0f0f687e0c6783348ea38911cb62c6c62c1e0f276233bcaa38817f7c95c"}
Sep 09 13:26:43.704857 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.704830    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bxvwp" event={"ID":"2568ae2b-2def-4a07-bb76-298b5a88c049","Type":"ContainerStarted","Data":"3020e63c069f85912471e0dcfbfd7655c89f8b318159c8ec971b9189e4580d5d"}
Sep 09 13:26:43.746888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:43.746843    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-v2xgb" podStartSLOduration=5.477055328 podStartE2EDuration="22.74682826s" totalImagesPullingTime="17.269772932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:42.905305302 +0000 UTC m=+21.794005294" watchObservedRunningTime="2026-09-09 13:26:43.74682826 +0000 UTC m=+22.635528309"
Sep 09 13:26:44.710878 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.710844    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"f6263d3ea999597fd33a608e43c1b29be3c28eb2ba7377b798feb19dce788807"}
Sep 09 13:26:44.713268 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.713206    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5" event={"ID":"338cedfc-2d55-4430-9472-42e3415b4ba1","Type":"ContainerStarted","Data":"32c9433f4340215f67a14f9e9f6f9a501b7f9a13b8837b3528a8d843e6d0adce"}
Sep 09 13:26:44.764946 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.764909    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bxvwp" podStartSLOduration=6.420123376 podStartE2EDuration="23.764896166s" totalImagesPullingTime="17.34477279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:43.747093051 +0000 UTC m=+22.635793043" watchObservedRunningTime="2026-09-09 13:26:44.764896166 +0000 UTC m=+23.653596156"
Sep 09 13:26:44.898116 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.898078    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:44.967809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.967747    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:44.968306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.968286    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:44.994724 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:44.994686    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wx9b5" podStartSLOduration=4.485177109 podStartE2EDuration="23.994672749s" totalImagesPullingTime="19.50949564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:44.764820622 +0000 UTC m=+23.653520612" watchObservedRunningTime="2026-09-09 13:26:44.994672749 +0000 UTC m=+23.883372753"
Sep 09 13:26:45.610829 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:45.610798    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:45.611091 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:45.610798    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:45.611091 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:45.610898    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:45.611091 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:45.610986    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:45.611091 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:45.610796    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:45.611091 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:45.611062    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:45.716617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:45.716590    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xzxtz"
Sep 09 13:26:47.611085 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.610934    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:47.611761 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.610938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:47.611761 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:47.611152    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:47.611761 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.610947    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:47.611761 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:47.611242    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:47.611761 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:47.611305    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:47.723864 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.723834    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="feb08cdec358a35e927239bb4fbe1a7beef535c4c6981523e8b65aea8e1ba2eb" exitCode=0
Sep 09 13:26:47.723998 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.723907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"feb08cdec358a35e927239bb4fbe1a7beef535c4c6981523e8b65aea8e1ba2eb"}
Sep 09 13:26:47.727645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.727623    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" event={"ID":"8d2b4486-fdea-4c96-bdd2-cb0232ff47ce","Type":"ContainerStarted","Data":"d36d4d52c2409c42d5356139460586886f813dbe2ef87cc499db2eaa1b2d4bff"}
Sep 09 13:26:47.727857 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.727840    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:47.742789 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.742769    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:47.845807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:47.845768    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4" podStartSLOduration=9.509709576 podStartE2EDuration="26.845755373s" totalImagesPullingTime="17.336045797s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:47.845372837 +0000 UTC m=+26.734072840" watchObservedRunningTime="2026-09-09 13:26:47.845755373 +0000 UTC m=+26.734455412"
Sep 09 13:26:48.735272 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:48.735128    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerStarted","Data":"24f029d70e705e0b1f87e4cf6ba18bb09d9beb478f332e291de5bd6e6af42d77"}
Sep 09 13:26:48.735932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:48.735909    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:48.736031 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:48.735996    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:48.749559 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:48.749540    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:26:49.212686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.212599    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gv79z"]
Sep 09 13:26:49.212827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.212758    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:49.212887 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:49.212855    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:49.213223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.213204    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p5cph"]
Sep 09 13:26:49.213297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.213286    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:49.213387 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:49.213366    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:49.221623 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.221599    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nfsvq"]
Sep 09 13:26:49.221707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.221675    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:49.221755 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:49.221742    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:49.738508 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.738480    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="24f029d70e705e0b1f87e4cf6ba18bb09d9beb478f332e291de5bd6e6af42d77" exitCode=0
Sep 09 13:26:49.738945 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:49.738568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"24f029d70e705e0b1f87e4cf6ba18bb09d9beb478f332e291de5bd6e6af42d77"}
Sep 09 13:26:50.610910 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:50.610882    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:50.611037 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:50.611016    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:50.611093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:50.611061    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:50.611168 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:50.611149    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:50.743234 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:50.743203    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="8141ebc9991991c07f406bc9d8bfb6a78a0e67e5473f0d98a4ddd76aa0997059" exitCode=0
Sep 09 13:26:50.743609 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:50.743255    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"8141ebc9991991c07f406bc9d8bfb6a78a0e67e5473f0d98a4ddd76aa0997059"}
Sep 09 13:26:51.611124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:51.611090    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:51.611285 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:51.611193    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:52.611083 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:52.611053    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:52.611463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:52.611052    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:52.611463 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:52.611163    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:52.611463 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:52.611306    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:53.611229 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:53.611017    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:53.611688 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:53.611343    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv79z" podUID="e7315ff5-e3eb-421b-b3cd-f0460cb94ee4"
Sep 09 13:26:54.610745 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:54.610705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:54.610938 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:54.610757    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:54.610938 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:54.610904    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nfsvq" podUID="36eed273-3006-4797-978e-bf5d1f555385"
Sep 09 13:26:54.611059 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:54.610957    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-p5cph" podUID="efee7a0a-5265-4555-8c88-8ed8e7b6aef7"
Sep 09 13:26:54.916663 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:54.916632    2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-35.ec2.internal" event="NodeReady"
Sep 09 13:26:54.917195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:54.916757    2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 13:26:54.998034 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:54.997759    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:26:55.010306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.010288    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-qh9sn"]
Sep 09 13:26:55.010413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.010398    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.021240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.021219    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 13:26:55.021535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.021513    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 13:26:55.021658 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.021614    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 13:26:55.022115 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.022097    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4wzf7\""
Sep 09 13:26:55.030479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.030458    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-7kfr4"]
Sep 09 13:26:55.030622 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.030594    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.039239 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.039220    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 13:26:55.039333 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.039305    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 13:26:55.039333 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.039322    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 13:26:55.039607 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.039593    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kmkp6\""
Sep 09 13:26:55.043765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.043746    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 13:26:55.050638 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.050621    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qh9sn"]
Sep 09 13:26:55.050713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.050647    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:26:55.050713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.050710    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.053300 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.053277    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 13:26:55.053398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.053328    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 13:26:55.053398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.053353    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ckz5q\""
Sep 09 13:26:55.061915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.061897    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kfr4"]
Sep 09 13:26:55.167062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167013    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167038    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861fccec-5212-4e31-ba42-a620e1e8e4bb-tmp-dir\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.167062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167056    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/861fccec-5212-4e31-ba42-a620e1e8e4bb-config-volume\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.167215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167162    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gctz4\" (UniqueName: \"kubernetes.io/projected/304f489b-4b99-4ff1-b5ee-469281fbdc59-kube-api-access-gctz4\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.167302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167234    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/304f489b-4b99-4ff1-b5ee-469281fbdc59-cert\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.167362 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167362 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167325    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167362 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8vw2\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167451 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167371    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.167451 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167386    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.167451 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.167400    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5w48\" (UniqueName: \"kubernetes.io/projected/861fccec-5212-4e31-ba42-a620e1e8e4bb-kube-api-access-x5w48\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.267959 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.267934    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.267964    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gctz4\" (UniqueName: \"kubernetes.io/projected/304f489b-4b99-4ff1-b5ee-469281fbdc59-kube-api-access-gctz4\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.268062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.267991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268012    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/304f489b-4b99-4ff1-b5ee-469281fbdc59-cert\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.268062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268088    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268112    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8vw2\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268141    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268165    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268190    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5w48\" (UniqueName: \"kubernetes.io/projected/861fccec-5212-4e31-ba42-a620e1e8e4bb-kube-api-access-x5w48\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268219    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268265 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268243    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861fccec-5212-4e31-ba42-a620e1e8e4bb-tmp-dir\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268279    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268299    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/861fccec-5212-4e31-ba42-a620e1e8e4bb-config-volume\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268331    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268350    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:26:55.76832783 +0000 UTC m=+34.657027799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268387    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268409    2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268466    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs podName:36eed273-3006-4797-978e-bf5d1f555385 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:27.268450783 +0000 UTC m=+66.157150766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs") pod "network-metrics-daemon-nfsvq" (UID: "36eed273-3006-4797-978e-bf5d1f555385") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268480    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268494    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.268610 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.268554    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:26:55.768535799 +0000 UTC m=+34.657235770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:26:55.269162 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268675    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.269162 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.268777    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/861fccec-5212-4e31-ba42-a620e1e8e4bb-tmp-dir\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.269268 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.269173    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.269268 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.269212    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/861fccec-5212-4e31-ba42-a620e1e8e4bb-config-volume\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.272635 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.272605    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.272736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.272611    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.272736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.272636    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/304f489b-4b99-4ff1-b5ee-469281fbdc59-cert\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.282409 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.282377    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.283339 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.283312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gctz4\" (UniqueName: \"kubernetes.io/projected/304f489b-4b99-4ff1-b5ee-469281fbdc59-kube-api-access-gctz4\") pod \"ingress-canary-qh9sn\" (UID: \"304f489b-4b99-4ff1-b5ee-469281fbdc59\") " pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.283441 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.283394    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8vw2\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.284816 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.284796    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5w48\" (UniqueName: \"kubernetes.io/projected/861fccec-5212-4e31-ba42-a620e1e8e4bb-kube-api-access-x5w48\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.338643 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.338623    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-qh9sn"
Sep 09 13:26:55.368948 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.368927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:55.369026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.369003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:55.369082 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369067    2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:55.369117 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369102    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 13:26:55.369146 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369116    2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 13:26:55.369146 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369127    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret podName:e7315ff5-e3eb-421b-b3cd-f0460cb94ee4 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:27.369109728 +0000 UTC m=+66.257809712 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret") pod "global-pull-secret-syncer-gv79z" (UID: "e7315ff5-e3eb-421b-b3cd-f0460cb94ee4") : object "kube-system"/"original-pull-secret" not registered
Sep 09 13:26:55.369146 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369128    2559 projected.go:196] Error preparing data for projected volume kube-api-access-vdj2s for pod openshift-network-diagnostics/network-check-target-p5cph: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:55.369241 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.369163    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s podName:efee7a0a-5265-4555-8c88-8ed8e7b6aef7 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:27.369154712 +0000 UTC m=+66.257854693 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-vdj2s" (UniqueName: "kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s") pod "network-check-target-p5cph" (UID: "efee7a0a-5265-4555-8c88-8ed8e7b6aef7") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 13:26:55.611346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.611318    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:26:55.614962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.614943    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 13:26:55.674724 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.674546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-qh9sn"]
Sep 09 13:26:55.677609 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:55.677586    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod304f489b_4b99_4ff1_b5ee_469281fbdc59.slice/crio-91c3a9d56310016cad195805486a087f978541bfd0f305fa80936916d70b4dcc WatchSource:0}: Error finding container 91c3a9d56310016cad195805486a087f978541bfd0f305fa80936916d70b4dcc: Status 404 returned error can't find the container with id 91c3a9d56310016cad195805486a087f978541bfd0f305fa80936916d70b4dcc
Sep 09 13:26:55.758012 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.757980    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="7622cabeb9738ed3e44a00a8f6c43ef63c19af3000f1a55a3cca3f0ba211dc37" exitCode=0
Sep 09 13:26:55.758115 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.758044    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"7622cabeb9738ed3e44a00a8f6c43ef63c19af3000f1a55a3cca3f0ba211dc37"}
Sep 09 13:26:55.759247 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.759219    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qh9sn" event={"ID":"304f489b-4b99-4ff1-b5ee-469281fbdc59","Type":"ContainerStarted","Data":"91c3a9d56310016cad195805486a087f978541bfd0f305fa80936916d70b4dcc"}
Sep 09 13:26:55.771836 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.771813    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:55.771915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:55.771842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:55.771953 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.771931    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:26:55.771953 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.771937    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:26:55.772015 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.771981    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:26:56.77196922 +0000 UTC m=+35.660669188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:26:55.772065 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.771941    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:26:55.772065 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:55.772055    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:26:56.772044662 +0000 UTC m=+35.660744636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:26:56.610327 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.610254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:26:56.611135 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.610254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:26:56.614656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.614631    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 13:26:56.615167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.615144    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 13:26:56.615601 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.615566    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpj6j\""
Sep 09 13:26:56.615696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.615613    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 13:26:56.615696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.615659    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ps8jz\""
Sep 09 13:26:56.765584 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.765531    2559 generic.go:356] "Generic (PLEG): container finished" podID="cfb90b51-1423-4e1f-8771-82dd73ab0bce" containerID="977625108bdb7dbe794a6bb3692bf35bea176a3bc2cae3fb9fb8fbbe155a8d5f" exitCode=0
Sep 09 13:26:56.765738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.765592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerDied","Data":"977625108bdb7dbe794a6bb3692bf35bea176a3bc2cae3fb9fb8fbbe155a8d5f"}
Sep 09 13:26:56.780926 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.780900    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:56.781041 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:56.780937    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:56.781106 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:56.781044    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:26:56.781106 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:56.781083    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:26:56.781106 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:56.781097    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:26:56.781106 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:56.781103    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:26:58.781084034 +0000 UTC m=+37.669784006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:26:56.781314 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:56.781141    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:26:58.781127252 +0000 UTC m=+37.669827225 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:26:57.772126 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:57.771976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" event={"ID":"cfb90b51-1423-4e1f-8771-82dd73ab0bce","Type":"ContainerStarted","Data":"b77d1550e3b845db18b09be7a8f9df976ad21aee8e68488686076b4cc77a113e"}
Sep 09 13:26:57.825080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:57.825024    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xrlmc" podStartSLOduration=9.195114336 podStartE2EDuration="36.825002705s" totalImagesPullingTime="27.629888369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:57.824675066 +0000 UTC m=+36.713375058" watchObservedRunningTime="2026-09-09 13:26:57.825002705 +0000 UTC m=+36.713702695"
Sep 09 13:26:58.776319 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.776135    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-qh9sn" event={"ID":"304f489b-4b99-4ff1-b5ee-469281fbdc59","Type":"ContainerStarted","Data":"8a1178f413d1abcdd47b7ac96cd552fa065ef73c9172132234be23113714e642"}
Sep 09 13:26:58.800129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.800103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:26:58.800129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.800133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:26:58.800251 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:58.800229    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:26:58.800251 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:58.800243    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:26:58.800324 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:58.800229    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:26:58.800324 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:58.800294    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:27:02.800278762 +0000 UTC m=+41.688978740 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:26:58.800324 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:58.800306    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:27:02.800300024 +0000 UTC m=+41.688999992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:26:58.831759 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.831680    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-qh9sn" podStartSLOduration=2.438779927 podStartE2EDuration="4.831668087s" totalImagesPullingTime="2.39288816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:26:58.831443423 +0000 UTC m=+37.720143416" watchObservedRunningTime="2026-09-09 13:26:58.831668087 +0000 UTC m=+37.720368067"
Sep 09 13:26:58.861690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.861662    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zl4p7"]
Sep 09 13:26:58.884838 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.884816    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:58.890802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.890778    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 13:26:58.890924 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.890802    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 13:26:58.891105 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.891034    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zl4p7"]
Sep 09 13:26:58.891332 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.891189    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jg6wc\""
Sep 09 13:26:58.891332 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.891245    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 13:26:58.891738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:58.891721    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 13:26:59.001020 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.000988    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2626952-4ba1-4c76-a586-3764efb71eb1-crio-socket\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.001150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.001050    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2626952-4ba1-4c76-a586-3764efb71eb1-data-volume\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.001150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.001075    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvmvk\" (UniqueName: \"kubernetes.io/projected/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-api-access-mvmvk\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.001150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.001114    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.001281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.001171    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101478 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101419    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2626952-4ba1-4c76-a586-3764efb71eb1-data-volume\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101478 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101448    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvmvk\" (UniqueName: \"kubernetes.io/projected/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-api-access-mvmvk\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101617    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2626952-4ba1-4c76-a586-3764efb71eb1-crio-socket\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101660 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.101630    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 13:26:59.101823 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.101686    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:59.601672295 +0000 UTC m=+38.490372263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : configmap "kube-rbac-proxy" not found
Sep 09 13:26:59.101823 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.101761    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:26:59.101823 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101788    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f2626952-4ba1-4c76-a586-3764efb71eb1-data-volume\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.101951 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.101833    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:26:59.601817017 +0000 UTC m=+38.490516992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:26:59.101951 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.101849    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f2626952-4ba1-4c76-a586-3764efb71eb1-crio-socket\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.162426 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.162405    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvmvk\" (UniqueName: \"kubernetes.io/projected/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-api-access-mvmvk\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.360336 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.360266    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"]
Sep 09 13:26:59.379380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.379358    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"
Sep 09 13:26:59.383288 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.383264    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-vl4gr\""
Sep 09 13:26:59.383403 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.383326    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 13:26:59.384143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.384124    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"]
Sep 09 13:26:59.429474 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.429453    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 13:26:59.505337 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.505315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmmmg\" (UniqueName: \"kubernetes.io/projected/bd179afe-17f5-4660-b662-b5299d388a83-kube-api-access-fmmmg\") pod \"migrator-fdbb8fdff-p6kvg\" (UID: \"bd179afe-17f5-4660-b662-b5299d388a83\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"
Sep 09 13:26:59.606119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.606092    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmmmg\" (UniqueName: \"kubernetes.io/projected/bd179afe-17f5-4660-b662-b5299d388a83-kube-api-access-fmmmg\") pod \"migrator-fdbb8fdff-p6kvg\" (UID: \"bd179afe-17f5-4660-b662-b5299d388a83\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"
Sep 09 13:26:59.606221 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.606130    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.606221 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.606184    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:26:59.606304 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.606283    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 13:26:59.606304 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.606290    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:26:59.606431 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.606335    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:00.606320102 +0000 UTC m=+39.495020071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : configmap "kube-rbac-proxy" not found
Sep 09 13:26:59.606431 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:26:59.606351    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:00.606343797 +0000 UTC m=+39.495043765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:26:59.648699 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.648652    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmmmg\" (UniqueName: \"kubernetes.io/projected/bd179afe-17f5-4660-b662-b5299d388a83-kube-api-access-fmmmg\") pod \"migrator-fdbb8fdff-p6kvg\" (UID: \"bd179afe-17f5-4660-b662-b5299d388a83\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"
Sep 09 13:26:59.688788 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.688768    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"
Sep 09 13:26:59.974666 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:26:59.974636    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg"]
Sep 09 13:26:59.977751 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:26:59.977711    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd179afe_17f5_4660_b662_b5299d388a83.slice/crio-c30507749f8ec48c9c657c92a8db9d3737d88600832fe8379ac1f359f62cd5a6 WatchSource:0}: Error finding container c30507749f8ec48c9c657c92a8db9d3737d88600832fe8379ac1f359f62cd5a6: Status 404 returned error can't find the container with id c30507749f8ec48c9c657c92a8db9d3737d88600832fe8379ac1f359f62cd5a6
Sep 09 13:27:00.434929 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:00.434862    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2xgb_ffa1b527-a17f-4123-9ca0-33cf66176215/dns-node-resolver/0.log"
Sep 09 13:27:00.615473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:00.615433    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:00.615649 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:00.615527    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:00.615649 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:00.615614    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:00.615734 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:00.615648    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 13:27:00.615734 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:00.615682    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:02.615666481 +0000 UTC m=+41.504366449 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:00.615734 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:00.615701    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:02.615688773 +0000 UTC m=+41.504388741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : configmap "kube-rbac-proxy" not found
Sep 09 13:27:00.782214 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:00.782181    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg" event={"ID":"bd179afe-17f5-4660-b662-b5299d388a83","Type":"ContainerStarted","Data":"c30507749f8ec48c9c657c92a8db9d3737d88600832fe8379ac1f359f62cd5a6"}
Sep 09 13:27:00.977264 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:00.977235    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6zfw7_c75d4892-66db-4119-bd92-5b343db8b44e/node-ca/0.log"
Sep 09 13:27:01.775254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:01.775230    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qh9sn_304f489b-4b99-4ff1-b5ee-469281fbdc59/serve-healthcheck-canary/0.log"
Sep 09 13:27:02.019364 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.019332    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-25cml"]
Sep 09 13:27:02.051716 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.051655    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-25cml"]
Sep 09 13:27:02.051826 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.051741    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.058504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.058485    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 09 13:27:02.058625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.058524    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 09 13:27:02.059004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.058989    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 09 13:27:02.059597 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.059565    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-rxnqm\""
Sep 09 13:27:02.059666 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.059617    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 09 13:27:02.226550 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.226518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/191ef098-63c2-447d-abf4-517ab89cd37d-signing-key\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.226696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.226589    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/191ef098-63c2-447d-abf4-517ab89cd37d-signing-cabundle\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.226696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.226648    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r4p2\" (UniqueName: \"kubernetes.io/projected/191ef098-63c2-447d-abf4-517ab89cd37d-kube-api-access-9r4p2\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.327718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.327646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/191ef098-63c2-447d-abf4-517ab89cd37d-signing-key\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.327718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.327701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/191ef098-63c2-447d-abf4-517ab89cd37d-signing-cabundle\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.327962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.327748    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9r4p2\" (UniqueName: \"kubernetes.io/projected/191ef098-63c2-447d-abf4-517ab89cd37d-kube-api-access-9r4p2\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.328472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.328449    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/191ef098-63c2-447d-abf4-517ab89cd37d-signing-cabundle\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.330750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.330724    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/191ef098-63c2-447d-abf4-517ab89cd37d-signing-key\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.344940 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.344915    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r4p2\" (UniqueName: \"kubernetes.io/projected/191ef098-63c2-447d-abf4-517ab89cd37d-kube-api-access-9r4p2\") pod \"service-ca-75d99c484-25cml\" (UID: \"191ef098-63c2-447d-abf4-517ab89cd37d\") " pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.360656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.360635    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-25cml"
Sep 09 13:27:02.631048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.630980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:02.631048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.631031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:02.631215 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.631101    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 13:27:02.631215 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.631116    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:02.631215 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.631157    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:06.631143035 +0000 UTC m=+45.519843008 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : configmap "kube-rbac-proxy" not found
Sep 09 13:27:02.631215 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.631182    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:06.631166429 +0000 UTC m=+45.519866402 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:02.673134 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.673107    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-25cml"]
Sep 09 13:27:02.832017 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.831983    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:02.832195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:02.832038    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:02.832195 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.832154    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:27:02.832195 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.832178    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:27:02.832195 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.832150    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:27:02.832376 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.832244    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:27:10.832225153 +0000 UTC m=+49.720925128 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:27:02.832376 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:02.832262    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:27:10.832253312 +0000 UTC m=+49.720953282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:27:02.981529 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:02.981496    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod191ef098_63c2_447d_abf4_517ab89cd37d.slice/crio-f4d8475fa2ffb737691d801da841fffee7129a15f734d180e89d552990efb02e WatchSource:0}: Error finding container f4d8475fa2ffb737691d801da841fffee7129a15f734d180e89d552990efb02e: Status 404 returned error can't find the container with id f4d8475fa2ffb737691d801da841fffee7129a15f734d180e89d552990efb02e
Sep 09 13:27:03.791001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:03.790966    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-25cml" event={"ID":"191ef098-63c2-447d-abf4-517ab89cd37d","Type":"ContainerStarted","Data":"f4d8475fa2ffb737691d801da841fffee7129a15f734d180e89d552990efb02e"}
Sep 09 13:27:03.792440 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:03.792409    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg" event={"ID":"bd179afe-17f5-4660-b662-b5299d388a83","Type":"ContainerStarted","Data":"2829d0e59e67a594cf459bc73c01085f6e690dda9ebec2546ede9c0cb666c0d4"}
Sep 09 13:27:03.792531 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:03.792450    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg" event={"ID":"bd179afe-17f5-4660-b662-b5299d388a83","Type":"ContainerStarted","Data":"1955b1d69c4b42656c769a0e18032e760caea19a83957f5412eb80c0bf321436"}
Sep 09 13:27:03.824218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:03.824170    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-p6kvg" podStartSLOduration=1.473782025 podStartE2EDuration="4.824155359s" totalImagesPullingTime="3.350373334s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:03.822604993 +0000 UTC m=+42.711304982" watchObservedRunningTime="2026-09-09 13:27:03.824155359 +0000 UTC m=+42.712855349"
Sep 09 13:27:06.660044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:06.660011    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:06.660380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:06.660070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:06.660380 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:06.660144    2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 13:27:06.660380 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:06.660203    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:14.660189918 +0000 UTC m=+53.548889886 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : configmap "kube-rbac-proxy" not found
Sep 09 13:27:06.660380 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:06.660272    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:06.660380 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:06.660332    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:14.660319606 +0000 UTC m=+53.549019575 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:07.803124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:07.803091    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-25cml" event={"ID":"191ef098-63c2-447d-abf4-517ab89cd37d","Type":"ContainerStarted","Data":"a73941e2b5ce9b2ce247230e16b6d9e57a2e3af24c19db48fb66c01a014b0447"}
Sep 09 13:27:07.833757 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:07.833714    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-25cml" podStartSLOduration=2.7293923380000003 podStartE2EDuration="6.833700867s" totalImagesPullingTime="4.104308529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:07.833424738 +0000 UTC m=+46.722124728" watchObservedRunningTime="2026-09-09 13:27:07.833700867 +0000 UTC m=+46.722400856"
Sep 09 13:27:10.893566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:10.893533    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:10.893590    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:10.893673    2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:10.893714    2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:10.893725    2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-76d49cd44f-4w5rw: secret "image-registry-tls" not found
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:10.893732    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls podName:861fccec-5212-4e31-ba42-a620e1e8e4bb nodeName:}" failed. No retries permitted until 2026-09-09 13:27:26.893716778 +0000 UTC m=+65.782416746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls") pod "dns-default-7kfr4" (UID: "861fccec-5212-4e31-ba42-a620e1e8e4bb") : secret "dns-default-metrics-tls" not found
Sep 09 13:27:10.893952 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:10.893759    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls podName:39bc6b0c-e652-44ef-8aab-5744bdd5a2ea nodeName:}" failed. No retries permitted until 2026-09-09 13:27:26.893748721 +0000 UTC m=+65.782448689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls") pod "image-registry-76d49cd44f-4w5rw" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea") : secret "image-registry-tls" not found
Sep 09 13:27:14.722472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:14.722437    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:14.722863 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:14.722510    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:14.722863 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:14.722609    2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:14.722863 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:14.722673    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls podName:f2626952-4ba1-4c76-a586-3764efb71eb1 nodeName:}" failed. No retries permitted until 2026-09-09 13:27:30.722657544 +0000 UTC m=+69.611357517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zl4p7" (UID: "f2626952-4ba1-4c76-a586-3764efb71eb1") : secret "insights-runtime-extractor-tls" not found
Sep 09 13:27:14.722992 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:14.722971    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f2626952-4ba1-4c76-a586-3764efb71eb1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:20.753336 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:20.753308    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9tqg4"
Sep 09 13:27:26.912967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:26.912932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:26.912967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:26.912967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:26.915225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:26.915201    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/861fccec-5212-4e31-ba42-a620e1e8e4bb-metrics-tls\") pod \"dns-default-7kfr4\" (UID: \"861fccec-5212-4e31-ba42-a620e1e8e4bb\") " pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:26.915333 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:26.915319    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"image-registry-76d49cd44f-4w5rw\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") " pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:27.124160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.124132    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4wzf7\""
Sep 09 13:27:27.132247 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.132227    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:27.160719 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.160694    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ckz5q\""
Sep 09 13:27:27.168870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.168811    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:27.316506 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.316477    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:27:27.319297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.319279    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 13:27:27.329436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.329409    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/36eed273-3006-4797-978e-bf5d1f555385-metrics-certs\") pod \"network-metrics-daemon-nfsvq\" (UID: \"36eed273-3006-4797-978e-bf5d1f555385\") " pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:27:27.404822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.404803    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:27:27.407504 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:27.407481    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39bc6b0c_e652_44ef_8aab_5744bdd5a2ea.slice/crio-47291bc13c7e45d5b41979ed62a64f5aa7a63a561791e834f94615cf722500c4 WatchSource:0}: Error finding container 47291bc13c7e45d5b41979ed62a64f5aa7a63a561791e834f94615cf722500c4: Status 404 returned error can't find the container with id 47291bc13c7e45d5b41979ed62a64f5aa7a63a561791e834f94615cf722500c4
Sep 09 13:27:27.417775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.417755    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:27:27.417838 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.417796    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:27:27.420477 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.420453    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 13:27:27.420586 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.420507    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 13:27:27.429905 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.429884    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e7315ff5-e3eb-421b-b3cd-f0460cb94ee4-original-pull-secret\") pod \"global-pull-secret-syncer-gv79z\" (UID: \"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4\") " pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:27:27.430184 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.430170    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 13:27:27.439400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.439377    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-7kfr4"]
Sep 09 13:27:27.440694 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.440592    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdj2s\" (UniqueName: \"kubernetes.io/projected/efee7a0a-5265-4555-8c88-8ed8e7b6aef7-kube-api-access-vdj2s\") pod \"network-check-target-p5cph\" (UID: \"efee7a0a-5265-4555-8c88-8ed8e7b6aef7\") " pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:27:27.442252 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:27.442226    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod861fccec_5212_4e31_ba42_a620e1e8e4bb.slice/crio-4c40d935d02a992444df7f48a0c04d98cd7f80c1d0ccfbf8cbf37beaede7fdd1 WatchSource:0}: Error finding container 4c40d935d02a992444df7f48a0c04d98cd7f80c1d0ccfbf8cbf37beaede7fdd1: Status 404 returned error can't find the container with id 4c40d935d02a992444df7f48a0c04d98cd7f80c1d0ccfbf8cbf37beaede7fdd1
Sep 09 13:27:27.524312 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.524286    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-ps8jz\""
Sep 09 13:27:27.530290 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.530271    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zpj6j\""
Sep 09 13:27:27.532380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.532365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:27:27.538092 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.538072    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nfsvq"
Sep 09 13:27:27.719153 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.719125    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv79z"
Sep 09 13:27:27.811685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.811654    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-p5cph"]
Sep 09 13:27:27.814757 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.814729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nfsvq"]
Sep 09 13:27:27.815422 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:27.815398    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefee7a0a_5265_4555_8c88_8ed8e7b6aef7.slice/crio-b134fae4f679e65ede5bf1900552b0dd9b0b89866cb6ba067c645a27d78853e4 WatchSource:0}: Error finding container b134fae4f679e65ede5bf1900552b0dd9b0b89866cb6ba067c645a27d78853e4: Status 404 returned error can't find the container with id b134fae4f679e65ede5bf1900552b0dd9b0b89866cb6ba067c645a27d78853e4
Sep 09 13:27:27.819482 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:27.819457    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36eed273_3006_4797_978e_bf5d1f555385.slice/crio-c645f377d8f3be888e6d4063bc6c1c61efe3657a8ba52cc2e8dbd595f0b0e109 WatchSource:0}: Error finding container c645f377d8f3be888e6d4063bc6c1c61efe3657a8ba52cc2e8dbd595f0b0e109: Status 404 returned error can't find the container with id c645f377d8f3be888e6d4063bc6c1c61efe3657a8ba52cc2e8dbd595f0b0e109
Sep 09 13:27:27.858847 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.858816    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" event={"ID":"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea","Type":"ContainerStarted","Data":"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"}
Sep 09 13:27:27.858949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.858864    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" event={"ID":"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea","Type":"ContainerStarted","Data":"47291bc13c7e45d5b41979ed62a64f5aa7a63a561791e834f94615cf722500c4"}
Sep 09 13:27:27.858949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.858892    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:27.859952 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.859929    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p5cph" event={"ID":"efee7a0a-5265-4555-8c88-8ed8e7b6aef7","Type":"ContainerStarted","Data":"b134fae4f679e65ede5bf1900552b0dd9b0b89866cb6ba067c645a27d78853e4"}
Sep 09 13:27:27.860934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.860914    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nfsvq" event={"ID":"36eed273-3006-4797-978e-bf5d1f555385","Type":"ContainerStarted","Data":"c645f377d8f3be888e6d4063bc6c1c61efe3657a8ba52cc2e8dbd595f0b0e109"}
Sep 09 13:27:27.861995 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.861976    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kfr4" event={"ID":"861fccec-5212-4e31-ba42-a620e1e8e4bb","Type":"ContainerStarted","Data":"4c40d935d02a992444df7f48a0c04d98cd7f80c1d0ccfbf8cbf37beaede7fdd1"}
Sep 09 13:27:27.880646 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:27.880607    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" podStartSLOduration=79.880594549 podStartE2EDuration="1m19.880594549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:27.879707941 +0000 UTC m=+66.768407931" watchObservedRunningTime="2026-09-09 13:27:27.880594549 +0000 UTC m=+66.769294529"
Sep 09 13:27:28.006292 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:28.006263    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gv79z"]
Sep 09 13:27:28.008405 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:28.008377    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7315ff5_e3eb_421b_b3cd_f0460cb94ee4.slice/crio-79de740ec4779322f217be704498e424657cf8d7f14efde35d4a2603a9669fcc WatchSource:0}: Error finding container 79de740ec4779322f217be704498e424657cf8d7f14efde35d4a2603a9669fcc: Status 404 returned error can't find the container with id 79de740ec4779322f217be704498e424657cf8d7f14efde35d4a2603a9669fcc
Sep 09 13:27:28.868106 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:28.868066    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gv79z" event={"ID":"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4","Type":"ContainerStarted","Data":"79de740ec4779322f217be704498e424657cf8d7f14efde35d4a2603a9669fcc"}
Sep 09 13:27:29.874027 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.873975    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nfsvq" event={"ID":"36eed273-3006-4797-978e-bf5d1f555385","Type":"ContainerStarted","Data":"2ab112bcdf3e1d645925cd7c4e6703917f223fe7e8ee617ccdda38ccc39d2b6d"}
Sep 09 13:27:29.874501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.874035    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nfsvq" event={"ID":"36eed273-3006-4797-978e-bf5d1f555385","Type":"ContainerStarted","Data":"6b62bc6e0777e9ddf2fee14de2ecf84f3e51927bf5549f440a9f69be5e7a79db"}
Sep 09 13:27:29.875999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.875968    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kfr4" event={"ID":"861fccec-5212-4e31-ba42-a620e1e8e4bb","Type":"ContainerStarted","Data":"06a879252aa092f641fb8c0008da6493dbf7c2c4ad01cd75813c09736ef0f226"}
Sep 09 13:27:29.876119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.876015    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-7kfr4" event={"ID":"861fccec-5212-4e31-ba42-a620e1e8e4bb","Type":"ContainerStarted","Data":"c9a830a316a295d7c308123d51e769614e21561637201e2c771ae809cddd2007"}
Sep 09 13:27:29.876119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.876114    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:29.894215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.894167    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nfsvq" podStartSLOduration=67.424397411 podStartE2EDuration="1m8.89414793s" totalImagesPullingTime="1.469750519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:29.893013607 +0000 UTC m=+68.781713598" watchObservedRunningTime="2026-09-09 13:27:29.89414793 +0000 UTC m=+68.782847920"
Sep 09 13:27:29.910150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:29.910096    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-7kfr4" podStartSLOduration=34.062172971 podStartE2EDuration="35.91008006s" totalImagesPullingTime="1.847907089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:29.909896176 +0000 UTC m=+68.798596167" watchObservedRunningTime="2026-09-09 13:27:29.91008006 +0000 UTC m=+68.798780051"
Sep 09 13:27:30.685495 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:30.685449    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:27:30.745811 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:30.745776    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:30.748582 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:30.748521    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f2626952-4ba1-4c76-a586-3764efb71eb1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zl4p7\" (UID: \"f2626952-4ba1-4c76-a586-3764efb71eb1\") " pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:30.997615 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:30.997547    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jg6wc\""
Sep 09 13:27:31.005860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:31.005812    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zl4p7"
Sep 09 13:27:31.334068 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:31.333969    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zl4p7"]
Sep 09 13:27:31.339612 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:27:31.339549    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2626952_4ba1_4c76_a586_3764efb71eb1.slice/crio-04a29c0d5fc640cf20d1f27319f8d650d9dcd8adcaad81388416838f9a713a19 WatchSource:0}: Error finding container 04a29c0d5fc640cf20d1f27319f8d650d9dcd8adcaad81388416838f9a713a19: Status 404 returned error can't find the container with id 04a29c0d5fc640cf20d1f27319f8d650d9dcd8adcaad81388416838f9a713a19
Sep 09 13:27:31.887421 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:31.887376    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zl4p7" event={"ID":"f2626952-4ba1-4c76-a586-3764efb71eb1","Type":"ContainerStarted","Data":"70e4f297529aa995b6bbf1d3da14fdcef254effbc1c3ba757ae72043ff33cbfe"}
Sep 09 13:27:31.887638 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:31.887442    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zl4p7" event={"ID":"f2626952-4ba1-4c76-a586-3764efb71eb1","Type":"ContainerStarted","Data":"04a29c0d5fc640cf20d1f27319f8d650d9dcd8adcaad81388416838f9a713a19"}
Sep 09 13:27:34.899619 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.899560    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gv79z" event={"ID":"e7315ff5-e3eb-421b-b3cd-f0460cb94ee4","Type":"ContainerStarted","Data":"963b02ab9d8a2f572d74e9b2caec357ded6117c9c171f29d2554756ecc76918f"}
Sep 09 13:27:34.901360 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.901331    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-p5cph" event={"ID":"efee7a0a-5265-4555-8c88-8ed8e7b6aef7","Type":"ContainerStarted","Data":"e87e7fce5f66f78dc0580c363b3e6387475b96d3185f3c017e65358b60647cd1"}
Sep 09 13:27:34.901485 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.901459    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:27:34.903453 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.903428    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zl4p7" event={"ID":"f2626952-4ba1-4c76-a586-3764efb71eb1","Type":"ContainerStarted","Data":"459395a2d3b239c7ec578d8ca04f466a7a874aea7125c950b14c844d9f43e838"}
Sep 09 13:27:34.922206 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.922155    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gv79z" podStartSLOduration=66.862449041 podStartE2EDuration="1m12.922138747s" totalImagesPullingTime="6.059689706s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:34.920497162 +0000 UTC m=+73.809197153" watchObservedRunningTime="2026-09-09 13:27:34.922138747 +0000 UTC m=+73.810838737"
Sep 09 13:27:34.939672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:34.939630    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-p5cph" podStartSLOduration=67.707733354 podStartE2EDuration="1m13.939613862s" totalImagesPullingTime="6.231880508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:26:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:34.938186963 +0000 UTC m=+73.826886954" watchObservedRunningTime="2026-09-09 13:27:34.939613862 +0000 UTC m=+73.828313852"
Sep 09 13:27:35.911636 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:35.911602    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zl4p7" event={"ID":"f2626952-4ba1-4c76-a586-3764efb71eb1","Type":"ContainerStarted","Data":"ca510eacacb76210fc4f8820d8620debf47e2ddc670a7eb7e183fd4eae9606f7"}
Sep 09 13:27:35.946788 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:35.946729    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zl4p7" podStartSLOduration=34.036343443 podStartE2EDuration="37.946715095s" totalImagesPullingTime="3.910371652s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:26:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:27:35.94542623 +0000 UTC m=+74.834126244" watchObservedRunningTime="2026-09-09 13:27:35.946715095 +0000 UTC m=+74.835415085"
Sep 09 13:27:39.883596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:39.883547    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-7kfr4"
Sep 09 13:27:50.691764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:50.691652    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:55.711544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.711497    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" podUID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" containerName="registry" containerID="cri-o://ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae" gracePeriod=30
Sep 09 13:27:55.941219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.941195    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:55.975994 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.975925    2559 generic.go:356] "Generic (PLEG): container finished" podID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" containerID="ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae" exitCode=0
Sep 09 13:27:55.976114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.976012    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" event={"ID":"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea","Type":"ContainerDied","Data":"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"}
Sep 09 13:27:55.976114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.976041    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw"
Sep 09 13:27:55.976114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.976058    2559 scope.go:117] "RemoveContainer" containerID="ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"
Sep 09 13:27:55.976311 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.976046    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-76d49cd44f-4w5rw" event={"ID":"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea","Type":"ContainerDied","Data":"47291bc13c7e45d5b41979ed62a64f5aa7a63a561791e834f94615cf722500c4"}
Sep 09 13:27:55.984105 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.984078    2559 scope.go:117] "RemoveContainer" containerID="ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"
Sep 09 13:27:55.984375 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:27:55.984352    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae\": container with ID starting with ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae not found: ID does not exist" containerID="ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"
Sep 09 13:27:55.984455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:55.984381    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae"} err="failed to get container status \"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae\": rpc error: code = NotFound desc = could not find container \"ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae\": container with ID starting with ac4710f94dbb790f874dccfab6657dfdfc4dfa1ece719cb17ce39db32b9f3fae not found: ID does not exist"
Sep 09 13:27:56.013776 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.013753    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.013862 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.013791    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.013862 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.013820    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8vw2\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014010 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.013991    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014040    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014095 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014066    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014220 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014200    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014238    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates\") pod \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\" (UID: \"39bc6b0c-e652-44ef-8aab-5744bdd5a2ea\") "
Sep 09 13:27:56.014297 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014247    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:27:56.014466 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014412    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-trusted-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.014813 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.014783    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:27:56.016313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.016281    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:27:56.016408 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.016305    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2" (OuterVolumeSpecName: "kube-api-access-v8vw2") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "kube-api-access-v8vw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:27:56.016667 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.016641    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:27:56.016763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.016664    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:27:56.016763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.016734    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:27:56.022798 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.022776    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" (UID: "39bc6b0c-e652-44ef-8aab-5744bdd5a2ea"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:27:56.115522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115495    2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-bound-sa-token\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115520    2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-installation-pull-secrets\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115536    2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-image-registry-private-configuration\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115550    2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-ca-trust-extracted\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115564    2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-certificates\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115597    2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-registry-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.115686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.115610    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8vw2\" (UniqueName: \"kubernetes.io/projected/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea-kube-api-access-v8vw2\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:27:56.297615 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.297586    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:27:56.301189 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:56.301167    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-76d49cd44f-4w5rw"]
Sep 09 13:27:57.614639 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:27:57.614609    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" path="/var/lib/kubelet/pods/39bc6b0c-e652-44ef-8aab-5744bdd5a2ea/volumes"
Sep 09 13:28:00.530238 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.530202    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-98rb6"]
Sep 09 13:28:00.530721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.530417    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" containerName="registry"
Sep 09 13:28:00.530721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.530431    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" containerName="registry"
Sep 09 13:28:00.530721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.530477    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="39bc6b0c-e652-44ef-8aab-5744bdd5a2ea" containerName="registry"
Sep 09 13:28:00.535212 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.535193    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.538230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.538210    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 13:28:00.539294 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.539262    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 09 13:28:00.539400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.539346    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 09 13:28:00.539400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.539365    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 13:28:00.539508 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.539411    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-9l4s5\""
Sep 09 13:28:00.539508 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.539458    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 13:28:00.547910 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.547886    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-98rb6"]
Sep 09 13:28:00.643499 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.643467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7td2\" (UniqueName: \"kubernetes.io/projected/fd50e27d-7078-4e17-b6fd-9dd2352afba0-kube-api-access-g7td2\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.643661 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.643525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.643661 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.643560    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.643661 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.643615    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd50e27d-7078-4e17-b6fd-9dd2352afba0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.744223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.744181    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.744223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.744229    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.744380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.744257    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd50e27d-7078-4e17-b6fd-9dd2352afba0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.744380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.744293    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7td2\" (UniqueName: \"kubernetes.io/projected/fd50e27d-7078-4e17-b6fd-9dd2352afba0-kube-api-access-g7td2\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.744955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.744932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fd50e27d-7078-4e17-b6fd-9dd2352afba0-metrics-client-ca\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.746511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.746491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-tls\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.746600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.746507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fd50e27d-7078-4e17-b6fd-9dd2352afba0-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.752873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.752853    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7td2\" (UniqueName: \"kubernetes.io/projected/fd50e27d-7078-4e17-b6fd-9dd2352afba0-kube-api-access-g7td2\") pod \"prometheus-operator-6f4946db56-98rb6\" (UID: \"fd50e27d-7078-4e17-b6fd-9dd2352afba0\") " pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:00.843853 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:00.843796    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6"
Sep 09 13:28:01.110778 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:01.110708    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-6f4946db56-98rb6"]
Sep 09 13:28:01.113556 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:01.113530    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd50e27d_7078_4e17_b6fd_9dd2352afba0.slice/crio-0039f9b33650e501353b1063d9314da2e9d946a95f3205412228603be3aa7022 WatchSource:0}: Error finding container 0039f9b33650e501353b1063d9314da2e9d946a95f3205412228603be3aa7022: Status 404 returned error can't find the container with id 0039f9b33650e501353b1063d9314da2e9d946a95f3205412228603be3aa7022
Sep 09 13:28:01.997625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:01.997587    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6" event={"ID":"fd50e27d-7078-4e17-b6fd-9dd2352afba0","Type":"ContainerStarted","Data":"0039f9b33650e501353b1063d9314da2e9d946a95f3205412228603be3aa7022"}
Sep 09 13:28:03.003077 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:03.003042    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6" event={"ID":"fd50e27d-7078-4e17-b6fd-9dd2352afba0","Type":"ContainerStarted","Data":"95cee5876513c58d50e46f309e0be3ff18c865d483c08f38e9f06b29c66562e6"}
Sep 09 13:28:03.003456 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:03.003098    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6" event={"ID":"fd50e27d-7078-4e17-b6fd-9dd2352afba0","Type":"ContainerStarted","Data":"557a0c58ebfe1ba334ba6b8c05149d77fbf0e47cd14f6eba356cc8eedfc07591"}
Sep 09 13:28:03.021795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:03.021752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-6f4946db56-98rb6" podStartSLOduration=1.6923270769999998 podStartE2EDuration="3.021738909s" totalImagesPullingTime="1.329411832s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:03.020601926 +0000 UTC m=+101.909301910" watchObservedRunningTime="2026-09-09 13:28:03.021738909 +0000 UTC m=+101.910438899"
Sep 09 13:28:04.902193 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.902153    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"]
Sep 09 13:28:04.905824 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.905804    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.909633 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.909609    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 13:28:04.909776 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.909610    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 13:28:04.909776 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.909682    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-mqd85\""
Sep 09 13:28:04.909776 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.909615    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 13:28:04.921589 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.918863    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"]
Sep 09 13:28:04.923370 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.923347    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wz9g9"]
Sep 09 13:28:04.926868 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.926850    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.929384 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.929364    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 13:28:04.929505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.929433    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 13:28:04.929680 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.929430    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lqv6c\""
Sep 09 13:28:04.929785 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.929766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 13:28:04.974096 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974072    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974190 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974110    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.974190 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974131    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.974190 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974149    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.974190 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974172    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-tls\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-root\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974284    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-sys\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974315    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/85df1f1d-b27e-42b0-987b-30db495b9939-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974344    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-wtmp\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974413 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974401    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dslcn\" (UniqueName: \"kubernetes.io/projected/f929a2f6-b4fa-4d8e-86c5-20abd3764133-kube-api-access-dslcn\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974653 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974458    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-metrics-client-ca\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974653 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974514    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-textfile\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974653 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974538    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:04.974653 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:04.974617    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfw7t\" (UniqueName: \"kubernetes.io/projected/85df1f1d-b27e-42b0-987b-30db495b9939-kube-api-access-vfw7t\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075084    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfw7t\" (UniqueName: \"kubernetes.io/projected/85df1f1d-b27e-42b0-987b-30db495b9939-kube-api-access-vfw7t\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075116    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075439    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075540 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075486    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075540 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075517    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-tls\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-root\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-sys\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075619    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/85df1f1d-b27e-42b0-987b-30db495b9939-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075644    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075673 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-wtmp\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075701    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dslcn\" (UniqueName: \"kubernetes.io/projected/f929a2f6-b4fa-4d8e-86c5-20abd3764133-kube-api-access-dslcn\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075729    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-metrics-client-ca\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-textfile\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-accelerators-collector-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:28:05.075879    2559 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.075891    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.075958 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:28:05.075932    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls podName:85df1f1d-b27e-42b0-987b-30db495b9939 nodeName:}" failed. No retries permitted until 2026-09-09 13:28:05.575914439 +0000 UTC m=+104.464614413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-xxjbf" (UID: "85df1f1d-b27e-42b0-987b-30db495b9939") : secret "kube-state-metrics-tls" not found
Sep 09 13:28:05.076302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076103    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/85df1f1d-b27e-42b0-987b-30db495b9939-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.076302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076125    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-textfile\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.076302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076181    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-root\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.076302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-wtmp\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.076498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076382    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f929a2f6-b4fa-4d8e-86c5-20abd3764133-sys\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.076712 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076688    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/85df1f1d-b27e-42b0-987b-30db495b9939-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.076850 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.076746    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f929a2f6-b4fa-4d8e-86c5-20abd3764133-metrics-client-ca\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.078755 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.078732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.078871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.078852    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.079070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.079053    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f929a2f6-b4fa-4d8e-86c5-20abd3764133-node-exporter-tls\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.083048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.083021    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfw7t\" (UniqueName: \"kubernetes.io/projected/85df1f1d-b27e-42b0-987b-30db495b9939-kube-api-access-vfw7t\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.084657 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.084634    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dslcn\" (UniqueName: \"kubernetes.io/projected/f929a2f6-b4fa-4d8e-86c5-20abd3764133-kube-api-access-dslcn\") pod \"node-exporter-wz9g9\" (UID: \"f929a2f6-b4fa-4d8e-86c5-20abd3764133\") " pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.235180 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.235154    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wz9g9"
Sep 09 13:28:05.250901 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:05.250873    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf929a2f6_b4fa_4d8e_86c5_20abd3764133.slice/crio-ccacfa3ad7b19debdf9cb12bc4997a7d5eda0c6332bcd95fef8f452bbfa14460 WatchSource:0}: Error finding container ccacfa3ad7b19debdf9cb12bc4997a7d5eda0c6332bcd95fef8f452bbfa14460: Status 404 returned error can't find the container with id ccacfa3ad7b19debdf9cb12bc4997a7d5eda0c6332bcd95fef8f452bbfa14460
Sep 09 13:28:05.579349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.579270    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.581557 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.581528    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/85df1f1d-b27e-42b0-987b-30db495b9939-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-xxjbf\" (UID: \"85df1f1d-b27e-42b0-987b-30db495b9939\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.823526 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.823485    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"
Sep 09 13:28:05.917895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.917294    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-p5cph"
Sep 09 13:28:05.961165 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.960601    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:28:05.981149 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.981121    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:05.983453 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.983185    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:28:05.983980 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.983959    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 13:28:05.984098 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.984078    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.984460    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.984474    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.984756    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.984786    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.985038    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f67f9\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.985144    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.985174    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 13:28:05.985786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:05.985221    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 13:28:06.015669 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.015633    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz9g9" event={"ID":"f929a2f6-b4fa-4d8e-86c5-20abd3764133","Type":"ContainerStarted","Data":"ccacfa3ad7b19debdf9cb12bc4997a7d5eda0c6332bcd95fef8f452bbfa14460"}
Sep 09 13:28:06.082240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082245    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082266    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082289    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082321    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082397    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082490    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082514    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082628    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082665    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44kr2\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.082965 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.082730    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.126817 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.126792    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf"]
Sep 09 13:28:06.129170 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:06.129140    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85df1f1d_b27e_42b0_987b_30db495b9939.slice/crio-64872be63df74a7fbe1e6eb28a6fbf6b8535eaff50da183edb4f8fbbb569572b WatchSource:0}: Error finding container 64872be63df74a7fbe1e6eb28a6fbf6b8535eaff50da183edb4f8fbbb569572b: Status 404 returned error can't find the container with id 64872be63df74a7fbe1e6eb28a6fbf6b8535eaff50da183edb4f8fbbb569572b
Sep 09 13:28:06.183694 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183651    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.183855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183703    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44kr2\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.183855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183759    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.183855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183786    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.183855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.183855 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183842    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.184112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183865    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.184112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183894    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.184112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.183959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.184112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.184106    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.184289 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.184148    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.185114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.185049    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.185374 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.185347    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.185447 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.185394    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.185785 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.185349    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.186879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.186858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.186968 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.186903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.187028 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.187008    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.187764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.187720    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.188395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.188374    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.188915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.188889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.189298 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.189259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.189460 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.189437    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.193681 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.193662    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44kr2\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.197461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.197444    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.199192 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.199174    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out\") pod \"alertmanager-main-0\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.294555 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.294517    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:28:06.821597 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:06.821560    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:28:06.898008 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:06.897979    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5061c0_f4fc_47ae_b8f5_3834b86ea090.slice/crio-56b310f20b210973f2380d9a6864954672c480596ee13f14e72f39aa1a176310 WatchSource:0}: Error finding container 56b310f20b210973f2380d9a6864954672c480596ee13f14e72f39aa1a176310: Status 404 returned error can't find the container with id 56b310f20b210973f2380d9a6864954672c480596ee13f14e72f39aa1a176310
Sep 09 13:28:07.021499 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:07.021456    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"56b310f20b210973f2380d9a6864954672c480596ee13f14e72f39aa1a176310"}
Sep 09 13:28:07.022917 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:07.022895    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf" event={"ID":"85df1f1d-b27e-42b0-987b-30db495b9939","Type":"ContainerStarted","Data":"64872be63df74a7fbe1e6eb28a6fbf6b8535eaff50da183edb4f8fbbb569572b"}
Sep 09 13:28:08.028463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:08.028424    2559 generic.go:356] "Generic (PLEG): container finished" podID="f929a2f6-b4fa-4d8e-86c5-20abd3764133" containerID="2992f685c6af8cd23b1c34b4d347ca1d9ae2d1a815b5e60dda2a62963c08ea63" exitCode=0
Sep 09 13:28:08.028849 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:08.028513    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz9g9" event={"ID":"f929a2f6-b4fa-4d8e-86c5-20abd3764133","Type":"ContainerDied","Data":"2992f685c6af8cd23b1c34b4d347ca1d9ae2d1a815b5e60dda2a62963c08ea63"}
Sep 09 13:28:09.033735 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.033692    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz9g9" event={"ID":"f929a2f6-b4fa-4d8e-86c5-20abd3764133","Type":"ContainerStarted","Data":"cf92563719575e9496612f0ddf612513fac06f0121a0e68f224e01dd3358ff49"}
Sep 09 13:28:09.034077 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.033759    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wz9g9" event={"ID":"f929a2f6-b4fa-4d8e-86c5-20abd3764133","Type":"ContainerStarted","Data":"5d3610ca98c84170831f4ede8d69feeae982618bf72d5732bcb24d7c8c8be1ca"}
Sep 09 13:28:09.035860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.035837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf" event={"ID":"85df1f1d-b27e-42b0-987b-30db495b9939","Type":"ContainerStarted","Data":"4f5e74e57c7b7deb86355da30a7943ae355827c534994eb32121a66751847998"}
Sep 09 13:28:09.035950 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.035892    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf" event={"ID":"85df1f1d-b27e-42b0-987b-30db495b9939","Type":"ContainerStarted","Data":"a01df2cd5adf86e4340ed316b348c7237e08e3a4d62b81817671dfdc4b01672d"}
Sep 09 13:28:09.035950 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.035923    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf" event={"ID":"85df1f1d-b27e-42b0-987b-30db495b9939","Type":"ContainerStarted","Data":"af7f06bc6f6227d95485569c4c9af2ba50b28dbaafcefd2bb10996a011da581c"}
Sep 09 13:28:09.037244 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.037222    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec" exitCode=0
Sep 09 13:28:09.037316 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.037263    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec"}
Sep 09 13:28:09.053617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.053533    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wz9g9" podStartSLOduration=3.296362126 podStartE2EDuration="5.053515846s" totalImagesPullingTime="1.669582512s" totalInitContainerRuntime="87.571208ms" isStatefulPod=true podCreationTimestamp="2026-09-09 13:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:09.052907861 +0000 UTC m=+107.941607850" watchObservedRunningTime="2026-09-09 13:28:09.053515846 +0000 UTC m=+107.942215838"
Sep 09 13:28:09.104747 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.104702    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-xxjbf" podStartSLOduration=2.78644451 podStartE2EDuration="5.104689781s" totalImagesPullingTime="2.318245271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:09.104319964 +0000 UTC m=+107.993019956" watchObservedRunningTime="2026-09-09 13:28:09.104689781 +0000 UTC m=+107.993389771"
Sep 09 13:28:09.282744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.282707    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-66648567fc-842m5"]
Sep 09 13:28:09.304650 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.304564    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66648567fc-842m5"]
Sep 09 13:28:09.304776 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.304701    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.307552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307528    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-95fajic73364k\""
Sep 09 13:28:09.307687 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307564    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xqprw\""
Sep 09 13:28:09.307803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307777    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 09 13:28:09.307888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307825    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 09 13:28:09.307888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307841    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 09 13:28:09.307996 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.307900    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 09 13:28:09.414111 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414083    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-client-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414125    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-client-certs\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414169    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414188    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-audit-log\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414206    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-metrics-server-audit-profiles\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414224    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kzr8\" (UniqueName: \"kubernetes.io/projected/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-kube-api-access-9kzr8\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.414406 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.414283    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-tls\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515236 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515207    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-client-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515248    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-client-certs\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515332    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-audit-log\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515368    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-metrics-server-audit-profiles\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515613 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515409    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kzr8\" (UniqueName: \"kubernetes.io/projected/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-kube-api-access-9kzr8\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515613 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515449    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-tls\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.515785 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.515761    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-audit-log\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.516055 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.516032    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.516318 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.516300    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-metrics-server-audit-profiles\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.517833 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.517812    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-client-ca-bundle\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.517833 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.517835    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-tls\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.517959 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.517887    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-secret-metrics-server-client-certs\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.523164 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.523140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kzr8\" (UniqueName: \"kubernetes.io/projected/1a833e65-61dd-42c8-9303-a3db7e2b3d3f-kube-api-access-9kzr8\") pod \"metrics-server-66648567fc-842m5\" (UID: \"1a833e65-61dd-42c8-9303-a3db7e2b3d3f\") " pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.613425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.613355    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:09.677306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.677278    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"]
Sep 09 13:28:09.679878 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.679855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:09.682898 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.682700    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 13:28:09.683026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.683005    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-9mz4z\""
Sep 09 13:28:09.693945 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.693859    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"]
Sep 09 13:28:09.817961 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.817931    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j787k\" (UniqueName: \"kubernetes.io/projected/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-kube-api-access-j787k\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:09.818138 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.817973    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:09.900753 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.900684    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-66648567fc-842m5"]
Sep 09 13:28:09.903615 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:09.903587    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a833e65_61dd_42c8_9303_a3db7e2b3d3f.slice/crio-56a4c3e5ae3f7ab67e8e74201ea04c819da31b821d590793a4816640042aeb3a WatchSource:0}: Error finding container 56a4c3e5ae3f7ab67e8e74201ea04c819da31b821d590793a4816640042aeb3a: Status 404 returned error can't find the container with id 56a4c3e5ae3f7ab67e8e74201ea04c819da31b821d590793a4816640042aeb3a
Sep 09 13:28:09.918787 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.918760    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j787k\" (UniqueName: \"kubernetes.io/projected/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-kube-api-access-j787k\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:09.918879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.918812    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:09.918975 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:28:09.918952    2559 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Sep 09 13:28:09.919089 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:28:09.919023    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert podName:9edf72c6-c3ec-469e-926f-2adfcfd9b0a0 nodeName:}" failed. No retries permitted until 2026-09-09 13:28:10.419001793 +0000 UTC m=+109.307701768 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert") pod "monitoring-plugin-85bbbbbf4-c4bbt" (UID: "9edf72c6-c3ec-469e-926f-2adfcfd9b0a0") : secret "monitoring-plugin-cert" not found
Sep 09 13:28:09.931184 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:09.931163    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j787k\" (UniqueName: \"kubernetes.io/projected/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-kube-api-access-j787k\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:10.043332 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.043297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66648567fc-842m5" event={"ID":"1a833e65-61dd-42c8-9303-a3db7e2b3d3f","Type":"ContainerStarted","Data":"56a4c3e5ae3f7ab67e8e74201ea04c819da31b821d590793a4816640042aeb3a"}
Sep 09 13:28:10.109992 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.109962    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"]
Sep 09 13:28:10.112480 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.112458    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.115063 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115041    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 13:28:10.115173 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115070    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 13:28:10.115173 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115115    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 13:28:10.115357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115337    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bslv6\""
Sep 09 13:28:10.115557 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115540    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 13:28:10.115639 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.115547    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 13:28:10.121133 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.120982    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 13:28:10.128191 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.128167    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"]
Sep 09 13:28:10.221020 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.220978    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-serving-certs-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221168 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-federate-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221168 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221118    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221191    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-metrics-client-ca\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221241    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221280    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl4nb\" (UniqueName: \"kubernetes.io/projected/eb4d44ff-a5f9-4367-b151-3616768e602c-kube-api-access-fl4nb\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221335    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.221422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.221393    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.321974    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322021    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322057    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-serving-certs-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322083    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-federate-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322371 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322180    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-metrics-client-ca\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322371 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322222    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.322371 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322255    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl4nb\" (UniqueName: \"kubernetes.io/projected/eb4d44ff-a5f9-4367-b151-3616768e602c-kube-api-access-fl4nb\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.323008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.322937    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-serving-certs-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.323196 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.323145    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.323449 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.323406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb4d44ff-a5f9-4367-b151-3616768e602c-metrics-client-ca\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.325109 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.325080    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-telemeter-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.325192 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.325130    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.325227 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.325200    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-secret-telemeter-client\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.325306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.325284    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb4d44ff-a5f9-4367-b151-3616768e602c-federate-client-tls\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.332804 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.332785    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl4nb\" (UniqueName: \"kubernetes.io/projected/eb4d44ff-a5f9-4367-b151-3616768e602c-kube-api-access-fl4nb\") pod \"telemeter-client-5d58bc97cc-wpnt4\" (UID: \"eb4d44ff-a5f9-4367-b151-3616768e602c\") " pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.422769 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.422737    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"
Sep 09 13:28:10.422967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.422939    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:10.425612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.425589    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9edf72c6-c3ec-469e-926f-2adfcfd9b0a0-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-c4bbt\" (UID: \"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:10.592875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.592855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:10.780825 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.780791    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4"]
Sep 09 13:28:10.786081 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:10.786051    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4d44ff_a5f9_4367_b151_3616768e602c.slice/crio-57fb54695826b3daec2d352129d1ce4776b457ee6e06adf9852da38aa99f2ad1 WatchSource:0}: Error finding container 57fb54695826b3daec2d352129d1ce4776b457ee6e06adf9852da38aa99f2ad1: Status 404 returned error can't find the container with id 57fb54695826b3daec2d352129d1ce4776b457ee6e06adf9852da38aa99f2ad1
Sep 09 13:28:10.868906 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:10.868827    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"]
Sep 09 13:28:10.872527 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:10.872483    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9edf72c6_c3ec_469e_926f_2adfcfd9b0a0.slice/crio-df25f3f158cbe089cac2b36f9935623265605a526e7ddbb5cbd1416f1e11f593 WatchSource:0}: Error finding container df25f3f158cbe089cac2b36f9935623265605a526e7ddbb5cbd1416f1e11f593: Status 404 returned error can't find the container with id df25f3f158cbe089cac2b36f9935623265605a526e7ddbb5cbd1416f1e11f593
Sep 09 13:28:11.051188 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.051106    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4" event={"ID":"eb4d44ff-a5f9-4367-b151-3616768e602c","Type":"ContainerStarted","Data":"57fb54695826b3daec2d352129d1ce4776b457ee6e06adf9852da38aa99f2ad1"}
Sep 09 13:28:11.052624 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.052592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt" event={"ID":"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0","Type":"ContainerStarted","Data":"df25f3f158cbe089cac2b36f9935623265605a526e7ddbb5cbd1416f1e11f593"}
Sep 09 13:28:11.056278 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.056256    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"}
Sep 09 13:28:11.056373 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.056306    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"}
Sep 09 13:28:11.056373 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.056332    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"}
Sep 09 13:28:11.056373 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.056356    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"}
Sep 09 13:28:11.056472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:11.056378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"}
Sep 09 13:28:12.064926 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:12.064837    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-66648567fc-842m5" event={"ID":"1a833e65-61dd-42c8-9303-a3db7e2b3d3f","Type":"ContainerStarted","Data":"6e755c58adfc13dc90c28ae9ed87eed81101ac3af5429cabe75c182efb30e19c"}
Sep 09 13:28:12.087256 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:12.087199    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-66648567fc-842m5" podStartSLOduration=1.183101766 podStartE2EDuration="3.087179104s" totalImagesPullingTime="1.904077338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:12.085350354 +0000 UTC m=+110.974050348" watchObservedRunningTime="2026-09-09 13:28:12.087179104 +0000 UTC m=+110.975879095"
Sep 09 13:28:14.075953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.075911    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerStarted","Data":"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"}
Sep 09 13:28:14.077881 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.077854    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4" event={"ID":"eb4d44ff-a5f9-4367-b151-3616768e602c","Type":"ContainerStarted","Data":"25bf7a89b15b3853e62abe6384a52692d966719343a475a9d4345877f5805a8b"}
Sep 09 13:28:14.077993 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.077910    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4" event={"ID":"eb4d44ff-a5f9-4367-b151-3616768e602c","Type":"ContainerStarted","Data":"34c31529938eb01a51b88e94ff0ab4dda8a1a67d52eee293568a76d2369a4129"}
Sep 09 13:28:14.077993 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.077939    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4" event={"ID":"eb4d44ff-a5f9-4367-b151-3616768e602c","Type":"ContainerStarted","Data":"89fb1074659a64c80b014935be3131eb3ff2cf806aec9219962871bea1d74d99"}
Sep 09 13:28:14.081812 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.079588    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt" event={"ID":"9edf72c6-c3ec-469e-926f-2adfcfd9b0a0","Type":"ContainerStarted","Data":"04150706c1d40c1e510a582e5db6faa486ea19b8146e59cdab910811c2fa39bf"}
Sep 09 13:28:14.081812 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.080238    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:14.086300 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.086280    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt"
Sep 09 13:28:14.104874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.104770    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.383007047 podStartE2EDuration="9.104755892s" totalImagesPullingTime="5.695970839s" totalInitContainerRuntime="25.778006ms" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:05 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:14.103049175 +0000 UTC m=+112.991749164" watchObservedRunningTime="2026-09-09 13:28:14.104755892 +0000 UTC m=+112.993455882"
Sep 09 13:28:14.121305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.121264    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-c4bbt" podStartSLOduration=2.841408874 podStartE2EDuration="5.121250579s" totalImagesPullingTime="2.279841705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:14.121098468 +0000 UTC m=+113.009798458" watchObservedRunningTime="2026-09-09 13:28:14.121250579 +0000 UTC m=+113.009950568"
Sep 09 13:28:14.143045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:14.143004    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5d58bc97cc-wpnt4" podStartSLOduration=1.778858321 podStartE2EDuration="4.142991371s" totalImagesPullingTime="2.36413305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:14.142010014 +0000 UTC m=+113.030710004" watchObservedRunningTime="2026-09-09 13:28:14.142991371 +0000 UTC m=+113.031691361"
Sep 09 13:28:15.620751 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.620719    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:28:15.623941 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.623924    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.626701 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.626680    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 13:28:15.626822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.626723    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 13:28:15.626822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.626740    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 13:28:15.626822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.626764    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 13:28:15.628204 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.628189    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 13:28:15.628282 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.628232    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 13:28:15.628282 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.628188    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-277tr\""
Sep 09 13:28:15.628393 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.628304    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 13:28:15.632398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.632378    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 13:28:15.634746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.634729    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:28:15.772630 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772601    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772770 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772633    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772770 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772688    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772770 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772712    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772827    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.772861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.772858    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sffmk\" (UniqueName: \"kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.873962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.873893    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.873962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.873927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.873962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.873949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.874140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.874140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874061    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.874140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874079    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.874140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874114    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sffmk\" (UniqueName: \"kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.874917 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874885    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.875044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.875044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.874961    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.875044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.875003    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.876369 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.876345    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.876551 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.876531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.882520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.882499    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sffmk\" (UniqueName: \"kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk\") pod \"console-8587cd44f8-sf9ft\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") " pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:15.933916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:15.933886    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:16.209441 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:16.209416    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:28:16.211707 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:28:16.211679    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f172b9_2382_43e9_b758_4bc5491686c3.slice/crio-dcb529a1189078511842e7b9fbb5184f2865a29c566ee8be26168199d908e570 WatchSource:0}: Error finding container dcb529a1189078511842e7b9fbb5184f2865a29c566ee8be26168199d908e570: Status 404 returned error can't find the container with id dcb529a1189078511842e7b9fbb5184f2865a29c566ee8be26168199d908e570
Sep 09 13:28:17.095177 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:17.095138    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8587cd44f8-sf9ft" event={"ID":"06f172b9-2382-43e9-b758-4bc5491686c3","Type":"ContainerStarted","Data":"dcb529a1189078511842e7b9fbb5184f2865a29c566ee8be26168199d908e570"}
Sep 09 13:28:19.104742 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:19.104648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8587cd44f8-sf9ft" event={"ID":"06f172b9-2382-43e9-b758-4bc5491686c3","Type":"ContainerStarted","Data":"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"}
Sep 09 13:28:19.124128 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:19.124068    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8587cd44f8-sf9ft" podStartSLOduration=1.5534098790000002 podStartE2EDuration="4.124047389s" totalImagesPullingTime="2.57063751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:28:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:28:19.123302593 +0000 UTC m=+118.012002572" watchObservedRunningTime="2026-09-09 13:28:19.124047389 +0000 UTC m=+118.012747379"
Sep 09 13:28:25.935241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:25.935205    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:25.935731 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:25.935345    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:25.939809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:25.939789    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:26.138750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:26.138723    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:28:29.615660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:29.615619    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:29.615660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:29.615677    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:49.618791 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:49.618763    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:28:49.622542 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:28:49.622516    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-66648567fc-842m5"
Sep 09 13:29:25.276202 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276115    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:25.276713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276479    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="alertmanager" containerID="cri-o://643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9" gracePeriod=120
Sep 09 13:29:25.276713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276554    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-web" containerID="cri-o://b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571" gracePeriod=120
Sep 09 13:29:25.276713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276609    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="prom-label-proxy" containerID="cri-o://c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c" gracePeriod=120
Sep 09 13:29:25.276713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276619    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy" containerID="cri-o://83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc" gracePeriod=120
Sep 09 13:29:25.276713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276563    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="config-reloader" containerID="cri-o://ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b" gracePeriod=120
Sep 09 13:29:25.276963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:25.276533    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-metric" containerID="cri-o://e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011" gracePeriod=120
Sep 09 13:29:26.356313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356280    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c" exitCode=0
Sep 09 13:29:26.356313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356305    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc" exitCode=0
Sep 09 13:29:26.356313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356312    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571" exitCode=0
Sep 09 13:29:26.356313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356317    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b" exitCode=0
Sep 09 13:29:26.356313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356322    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9" exitCode=0
Sep 09 13:29:26.356807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356339    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"}
Sep 09 13:29:26.356807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356411    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"}
Sep 09 13:29:26.356807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356437    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"}
Sep 09 13:29:26.356807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356461    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"}
Sep 09 13:29:26.356807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.356484    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"}
Sep 09 13:29:26.514292 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.514260    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:26.583328 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583293    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44kr2\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583339    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583365    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583411    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583443    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583486    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583511    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583538    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583564    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583618    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583670    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.583727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583708    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.584060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583743    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls\") pod \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\" (UID: \"4b5061c0-f4fc-47ae-b8f5-3834b86ea090\") "
Sep 09 13:29:26.584060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583812    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:29:26.584060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.583867    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:29:26.584060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.584042    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.584263 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.584064    2559 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-alertmanager-main-db\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.586345 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.586306    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:29:26.586598 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.586547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2" (OuterVolumeSpecName: "kube-api-access-44kr2") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "kube-api-access-44kr2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:29:26.587458 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.587430    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.587609 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.587541    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.587715 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.587693    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.588090 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.588052    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:29:26.588590 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.588548    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume" (OuterVolumeSpecName: "config-volume") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.588829 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.588797    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.589313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.589289    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out" (OuterVolumeSpecName: "config-out") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:29:26.591671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.591646    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.598087 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.598061    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config" (OuterVolumeSpecName: "web-config") pod "4b5061c0-f4fc-47ae-b8f5-3834b86ea090" (UID: "4b5061c0-f4fc-47ae-b8f5-3834b86ea090"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:29:26.684617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684548    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684586    2559 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-out\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684597    2559 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-tls-assets\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684606    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684615    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-metrics-client-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684624    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684633    2559 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-web-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684642    2559 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-secret-alertmanager-main-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684650    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44kr2\" (UniqueName: \"kubernetes.io/projected/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-kube-api-access-44kr2\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684657    2559 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-config-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:26.684810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:26.684665    2559 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/4b5061c0-f4fc-47ae-b8f5-3834b86ea090-cluster-tls-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:29:27.363731 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.363696    2559 generic.go:356] "Generic (PLEG): container finished" podID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerID="e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011" exitCode=0
Sep 09 13:29:27.364183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.363764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"}
Sep 09 13:29:27.364183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.363781    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.364183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.363839    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"4b5061c0-f4fc-47ae-b8f5-3834b86ea090","Type":"ContainerDied","Data":"56b310f20b210973f2380d9a6864954672c480596ee13f14e72f39aa1a176310"}
Sep 09 13:29:27.364183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.363873    2559 scope.go:117] "RemoveContainer" containerID="c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"
Sep 09 13:29:27.376404 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.376385    2559 scope.go:117] "RemoveContainer" containerID="e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"
Sep 09 13:29:27.383051 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.383032    2559 scope.go:117] "RemoveContainer" containerID="83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"
Sep 09 13:29:27.388922 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.388898    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:27.390116 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.390098    2559 scope.go:117] "RemoveContainer" containerID="b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"
Sep 09 13:29:27.394394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.394373    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:27.396996 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.396857    2559 scope.go:117] "RemoveContainer" containerID="ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"
Sep 09 13:29:27.403221 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.403206    2559 scope.go:117] "RemoveContainer" containerID="643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"
Sep 09 13:29:27.409664 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.409645    2559 scope.go:117] "RemoveContainer" containerID="dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec"
Sep 09 13:29:27.416285 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.416268    2559 scope.go:117] "RemoveContainer" containerID="c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"
Sep 09 13:29:27.416540 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.416522    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c\": container with ID starting with c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c not found: ID does not exist" containerID="c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"
Sep 09 13:29:27.416628 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.416552    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c"} err="failed to get container status \"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c\": rpc error: code = NotFound desc = could not find container \"c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c\": container with ID starting with c9e2750d382900c6feae47e24676b7cf955b867c17ff2ec5055c33f09784244c not found: ID does not exist"
Sep 09 13:29:27.416628 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.416590    2559 scope.go:117] "RemoveContainer" containerID="e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"
Sep 09 13:29:27.416850 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.416832    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011\": container with ID starting with e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011 not found: ID does not exist" containerID="e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"
Sep 09 13:29:27.416888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.416857    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011"} err="failed to get container status \"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011\": rpc error: code = NotFound desc = could not find container \"e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011\": container with ID starting with e4cd5bb643742e4cdb25011a594939ac36f7ad0e3530604d8045e2f616bf9011 not found: ID does not exist"
Sep 09 13:29:27.416888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.416872    2559 scope.go:117] "RemoveContainer" containerID="83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"
Sep 09 13:29:27.417084 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.417068    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc\": container with ID starting with 83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc not found: ID does not exist" containerID="83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"
Sep 09 13:29:27.417138 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417093    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc"} err="failed to get container status \"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc\": rpc error: code = NotFound desc = could not find container \"83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc\": container with ID starting with 83b18a039e266482239b0057d35999fa0a17a2c49e0e34ef5edc5a72a51bd0bc not found: ID does not exist"
Sep 09 13:29:27.417138 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417115    2559 scope.go:117] "RemoveContainer" containerID="b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"
Sep 09 13:29:27.417331 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.417315    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571\": container with ID starting with b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571 not found: ID does not exist" containerID="b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"
Sep 09 13:29:27.417410 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417335    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571"} err="failed to get container status \"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571\": rpc error: code = NotFound desc = could not find container \"b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571\": container with ID starting with b32e7fff76fd8a7af0ce90670939330000755b209291c1541e02e8b08fac4571 not found: ID does not exist"
Sep 09 13:29:27.417410 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417349    2559 scope.go:117] "RemoveContainer" containerID="ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"
Sep 09 13:29:27.417545 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.417529    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b\": container with ID starting with ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b not found: ID does not exist" containerID="ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"
Sep 09 13:29:27.417620 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417551    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b"} err="failed to get container status \"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b\": rpc error: code = NotFound desc = could not find container \"ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b\": container with ID starting with ba309cb4be4d8549e73ce8f117be130d267a4bc244773aa45e38bd539045cb3b not found: ID does not exist"
Sep 09 13:29:27.417620 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417587    2559 scope.go:117] "RemoveContainer" containerID="643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"
Sep 09 13:29:27.417807 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.417786    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9\": container with ID starting with 643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9 not found: ID does not exist" containerID="643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"
Sep 09 13:29:27.417844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417811    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9"} err="failed to get container status \"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9\": rpc error: code = NotFound desc = could not find container \"643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9\": container with ID starting with 643d5e5ec1ad222ca7e03e2ff1159dcae0ff1b48673bf7f1c1adbe8b2a5f8ec9 not found: ID does not exist"
Sep 09 13:29:27.417844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.417823    2559 scope.go:117] "RemoveContainer" containerID="dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec"
Sep 09 13:29:27.418010 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:29:27.417994    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec\": container with ID starting with dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec not found: ID does not exist" containerID="dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec"
Sep 09 13:29:27.418054 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.418015    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec"} err="failed to get container status \"dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec\": rpc error: code = NotFound desc = could not find container \"dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec\": container with ID starting with dd2e5e7ecc2e2a9bb28cdf94874b4db9cb8a1e49da1eeb525848658e906112ec not found: ID does not exist"
Sep 09 13:29:27.421781 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.421762    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:27.422018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422007    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-metric"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422020    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-metric"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422028    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="prom-label-proxy"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422034    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="prom-label-proxy"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422043    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="alertmanager"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422048    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="alertmanager"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422056    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy"
Sep 09 13:29:27.422062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422061    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422068    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="init-config-reloader"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422073    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="init-config-reloader"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422081    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-web"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422086    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-web"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422108    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="config-reloader"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422114    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="config-reloader"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422158    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="prom-label-proxy"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422165    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="config-reloader"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422170    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-metric"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422176    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="alertmanager"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422181    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy-web"
Sep 09 13:29:27.422261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.422187    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" containerName="kube-rbac-proxy"
Sep 09 13:29:27.427454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.427435    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.430288 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430273    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 13:29:27.430428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430407    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 13:29:27.430491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430412    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 13:29:27.430491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430470    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 13:29:27.430596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430499    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 13:29:27.430795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430778    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 13:29:27.430795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430788    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-f67f9\""
Sep 09 13:29:27.430915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430781    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 13:29:27.430915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.430787    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 13:29:27.435913 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.435853    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 13:29:27.440992 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.440970    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:27.490121 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-config-out\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490168    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490196    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490309 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490237    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490309 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490309 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490286    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490309 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490307    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-config-volume\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf64x\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-kube-api-access-sf64x\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490391    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490431    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490456    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.490482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.490471    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-web-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591058 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591037    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591101    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591118    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-web-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591135    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591151    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-config-out\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591199    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.591516 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591497    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591813    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591815    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591882    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591926    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-config-volume\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592146 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.591952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf64x\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-kube-api-access-sf64x\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.592809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.592784    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03b6c040-99cd-4055-90d5-cdf8620234d8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.594241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.594200    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/03b6c040-99cd-4055-90d5-cdf8620234d8-config-out\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.594331 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.594247    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.594388 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.594359    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.594708    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.594830    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.595016    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.595034    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595318 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.595099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-web-config\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.595762 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.595745    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/03b6c040-99cd-4055-90d5-cdf8620234d8-config-volume\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.600326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.600305    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf64x\" (UniqueName: \"kubernetes.io/projected/03b6c040-99cd-4055-90d5-cdf8620234d8-kube-api-access-sf64x\") pod \"alertmanager-main-0\" (UID: \"03b6c040-99cd-4055-90d5-cdf8620234d8\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:27.615767 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.615718    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b5061c0-f4fc-47ae-b8f5-3834b86ea090" path="/var/lib/kubelet/pods/4b5061c0-f4fc-47ae-b8f5-3834b86ea090/volumes"
Sep 09 13:29:27.737879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:27.737855    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 13:29:28.017831 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:28.017805    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 13:29:28.019816 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:29:28.019787    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b6c040_99cd_4055_90d5_cdf8620234d8.slice/crio-88b563a0e645e062a3489173333fa35d3bc451dbd5823dc7970cab660c152713 WatchSource:0}: Error finding container 88b563a0e645e062a3489173333fa35d3bc451dbd5823dc7970cab660c152713: Status 404 returned error can't find the container with id 88b563a0e645e062a3489173333fa35d3bc451dbd5823dc7970cab660c152713
Sep 09 13:29:28.369915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:28.369832    2559 generic.go:356] "Generic (PLEG): container finished" podID="03b6c040-99cd-4055-90d5-cdf8620234d8" containerID="8032fdba85b0c5b93330c590d359dfeca7201af98ee7b6566fedc5bf51abf281" exitCode=0
Sep 09 13:29:28.370330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:28.369926    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerDied","Data":"8032fdba85b0c5b93330c590d359dfeca7201af98ee7b6566fedc5bf51abf281"}
Sep 09 13:29:28.370330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:28.370050    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"88b563a0e645e062a3489173333fa35d3bc451dbd5823dc7970cab660c152713"}
Sep 09 13:29:29.380411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380378    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"e0abccbdf2390b32b13c4b0207e668ff8357e8ce475288572c71eb3173cad40e"}
Sep 09 13:29:29.380803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380441    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"5b9fb247848cbcb93de6382045d4bfffbf3cc003ff5882698d65e5cb59bf0e39"}
Sep 09 13:29:29.380803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380475    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"1d9a603637c8ab5f61c65d7c00b7a070b571ee687cd250fe891e6aa3d0a4e610"}
Sep 09 13:29:29.380803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380498    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"fe66dca0496dba75c33e29289a7555b71d88c2f2f4085448d13e9248dbd17875"}
Sep 09 13:29:29.380803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380520    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"c5af876773b6f6721e18bf03aaa1c1bbabcd70ff021982e9e732b1d741b245d0"}
Sep 09 13:29:29.380803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:29.380545    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"03b6c040-99cd-4055-90d5-cdf8620234d8","Type":"ContainerStarted","Data":"840316f3e7ebdc5491079708edd2b37216c9b284d0dd831e2cc23ac1895365bd"}
Sep 09 13:29:36.045583 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.045514    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=9.027763309 podStartE2EDuration="9.045494159s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.73085ms" isStatefulPod=false podCreationTimestamp="2026-09-09 13:29:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:29:29.411278568 +0000 UTC m=+188.299978583" watchObservedRunningTime="2026-09-09 13:29:36.045494159 +0000 UTC m=+194.934194149"
Sep 09 13:29:36.046036 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.045977    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:29:36.049443 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.049427    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.062600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.060557    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:29:36.161747 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161714    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hj7fw\" (UniqueName: \"kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.161895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161753    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.161895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161782    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.161895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161843    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.162021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161894    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.162021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161916    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.162021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.161979    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262406 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262376    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262413    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262433    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262450    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262489    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262516    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hj7fw\" (UniqueName: \"kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.262535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.262531    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.263267 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.263238    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.263385 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.263277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.263385 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.263326    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.263480 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.263379    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.264903 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.264879    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.264977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.264906    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.271112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.271094    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hj7fw\" (UniqueName: \"kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw\") pod \"console-6d48f9c9c7-vv249\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") " pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.360013 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.359962    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:36.629355 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:36.629330    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:29:36.631131 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:29:36.631104    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a97cbdc_03db_470c_8e25_ad7640a9a71c.slice/crio-e196af9b4e7b540bbad165541b2c7a26649a359a9b37fc3eb08529fa9e62c7f8 WatchSource:0}: Error finding container e196af9b4e7b540bbad165541b2c7a26649a359a9b37fc3eb08529fa9e62c7f8: Status 404 returned error can't find the container with id e196af9b4e7b540bbad165541b2c7a26649a359a9b37fc3eb08529fa9e62c7f8
Sep 09 13:29:37.419625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:37.419585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d48f9c9c7-vv249" event={"ID":"3a97cbdc-03db-470c-8e25-ad7640a9a71c","Type":"ContainerStarted","Data":"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"}
Sep 09 13:29:37.419966 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:37.419645    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d48f9c9c7-vv249" event={"ID":"3a97cbdc-03db-470c-8e25-ad7640a9a71c","Type":"ContainerStarted","Data":"e196af9b4e7b540bbad165541b2c7a26649a359a9b37fc3eb08529fa9e62c7f8"}
Sep 09 13:29:37.437830 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:37.437789    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6d48f9c9c7-vv249" podStartSLOduration=1.437773747 podStartE2EDuration="1.437773747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:29:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:29:37.437467598 +0000 UTC m=+196.326167589" watchObservedRunningTime="2026-09-09 13:29:37.437773747 +0000 UTC m=+196.326473736"
Sep 09 13:29:46.360321 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:46.360283    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:46.360710 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:46.360346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:46.364983 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:46.364960    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:46.455529 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:46.455510    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:29:46.503225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:29:46.503199    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:30:11.522564 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.522521    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8587cd44f8-sf9ft" podUID="06f172b9-2382-43e9-b758-4bc5491686c3" containerName="console" containerID="cri-o://00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d" gracePeriod=15
Sep 09 13:30:11.762238 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.762215    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:30:11.810349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810321    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810350    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810380    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sffmk\" (UniqueName: \"kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810425    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810477    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810699 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810502    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle\") pod \"06f172b9-2382-43e9-b758-4bc5491686c3\" (UID: \"06f172b9-2382-43e9-b758-4bc5491686c3\") "
Sep 09 13:30:11.810754 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810724    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:30:11.810861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810746    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config" (OuterVolumeSpecName: "console-config") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:30:11.810996 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.810922    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca" (OuterVolumeSpecName: "service-ca") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:30:11.811070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.811040    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:30:11.812594 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.812556    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:30:11.812689 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.812664    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:30:11.812732 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.812721    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk" (OuterVolumeSpecName: "kube-api-access-sffmk") pod "06f172b9-2382-43e9-b758-4bc5491686c3" (UID: "06f172b9-2382-43e9-b758-4bc5491686c3"). InnerVolumeSpecName "kube-api-access-sffmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:30:11.911933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911898    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-oauth-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.911933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911927    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-oauth-serving-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.911933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911938    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-console-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.912137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911947    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sffmk\" (UniqueName: \"kubernetes.io/projected/06f172b9-2382-43e9-b758-4bc5491686c3-kube-api-access-sffmk\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.912137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911956    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06f172b9-2382-43e9-b758-4bc5491686c3-console-serving-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.912137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911966    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:11.912137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:11.911974    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06f172b9-2382-43e9-b758-4bc5491686c3-trusted-ca-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:30:12.558203 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.558173    2559 generic.go:356] "Generic (PLEG): container finished" podID="06f172b9-2382-43e9-b758-4bc5491686c3" containerID="00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d" exitCode=0
Sep 09 13:30:12.558625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.558215    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8587cd44f8-sf9ft"
Sep 09 13:30:12.558625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.558262    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8587cd44f8-sf9ft" event={"ID":"06f172b9-2382-43e9-b758-4bc5491686c3","Type":"ContainerDied","Data":"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"}
Sep 09 13:30:12.558625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.558305    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8587cd44f8-sf9ft" event={"ID":"06f172b9-2382-43e9-b758-4bc5491686c3","Type":"ContainerDied","Data":"dcb529a1189078511842e7b9fbb5184f2865a29c566ee8be26168199d908e570"}
Sep 09 13:30:12.558625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.558326    2559 scope.go:117] "RemoveContainer" containerID="00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"
Sep 09 13:30:12.566891 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.566876    2559 scope.go:117] "RemoveContainer" containerID="00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"
Sep 09 13:30:12.567151 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:30:12.567132    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d\": container with ID starting with 00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d not found: ID does not exist" containerID="00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"
Sep 09 13:30:12.567201 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.567158    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d"} err="failed to get container status \"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d\": rpc error: code = NotFound desc = could not find container \"00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d\": container with ID starting with 00977551091e5de1cf2c5b89144ce9225330f5cda53ca2644191c601bd2d693d not found: ID does not exist"
Sep 09 13:30:12.579907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.579882    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:30:12.584220 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:12.584181    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8587cd44f8-sf9ft"]
Sep 09 13:30:13.615271 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:13.615236    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f172b9-2382-43e9-b758-4bc5491686c3" path="/var/lib/kubelet/pods/06f172b9-2382-43e9-b758-4bc5491686c3/volumes"
Sep 09 13:30:48.154274 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.154201    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"]
Sep 09 13:30:48.154711 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.154503    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f172b9-2382-43e9-b758-4bc5491686c3" containerName="console"
Sep 09 13:30:48.154711 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.154516    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f172b9-2382-43e9-b758-4bc5491686c3" containerName="console"
Sep 09 13:30:48.154711 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.154559    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f172b9-2382-43e9-b758-4bc5491686c3" containerName="console"
Sep 09 13:30:48.157482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.157466    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.160411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.160390    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 13:30:48.161586 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.161558    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 13:30:48.161666 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.161617    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8bfrp\""
Sep 09 13:30:48.165873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.165852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"]
Sep 09 13:30:48.269427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.269396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.269598 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.269466    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp7rd\" (UniqueName: \"kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.269598 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.269525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.370706 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.370670    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp7rd\" (UniqueName: \"kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.370706 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.370719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.370864 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.370743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.371141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.371122    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.371141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.371132    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.379818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.379790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp7rd\" (UniqueName: \"kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.466685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.466666    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:30:48.734275 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:48.734250    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"]
Sep 09 13:30:48.736674 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:30:48.736638    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87e90838_8a40_4dd0_8b07_846f0a2e8aa0.slice/crio-979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7 WatchSource:0}: Error finding container 979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7: Status 404 returned error can't find the container with id 979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7
Sep 09 13:30:49.690099 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:49.690052    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerStarted","Data":"979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7"}
Sep 09 13:30:54.712292 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:54.712252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerStarted","Data":"45bf3c7ef7e9f6b9c02eda465f6f0b40b5547bcc8433a3023d9fabf40455ed08"}
Sep 09 13:30:55.716729 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:55.716697    2559 generic.go:356] "Generic (PLEG): container finished" podID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerID="45bf3c7ef7e9f6b9c02eda465f6f0b40b5547bcc8433a3023d9fabf40455ed08" exitCode=0
Sep 09 13:30:55.717100 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:55.716743    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerDied","Data":"45bf3c7ef7e9f6b9c02eda465f6f0b40b5547bcc8433a3023d9fabf40455ed08"}
Sep 09 13:30:58.730418 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:58.730387    2559 generic.go:356] "Generic (PLEG): container finished" podID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerID="10e8059966dd42c1de30e160fef97247ad0d01ea8da3117643ef0af9b4c50dfc" exitCode=0
Sep 09 13:30:58.730757 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:30:58.730469    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerDied","Data":"10e8059966dd42c1de30e160fef97247ad0d01ea8da3117643ef0af9b4c50dfc"}
Sep 09 13:31:05.760754 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:05.760722    2559 generic.go:356] "Generic (PLEG): container finished" podID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerID="915e07d2756378122a0d00b493a6042f1d318f922ff9606deac9cd455ef34173" exitCode=0
Sep 09 13:31:05.761054 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:05.760771    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerDied","Data":"915e07d2756378122a0d00b493a6042f1d318f922ff9606deac9cd455ef34173"}
Sep 09 13:31:06.885821 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:06.885797    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:31:07.021606 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.021535    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle\") pod \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") "
Sep 09 13:31:07.021719 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.021613    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zp7rd\" (UniqueName: \"kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd\") pod \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") "
Sep 09 13:31:07.021719 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.021633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util\") pod \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\" (UID: \"87e90838-8a40-4dd0-8b07-846f0a2e8aa0\") "
Sep 09 13:31:07.022171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.022135    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle" (OuterVolumeSpecName: "bundle") pod "87e90838-8a40-4dd0-8b07-846f0a2e8aa0" (UID: "87e90838-8a40-4dd0-8b07-846f0a2e8aa0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:07.023690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.023662    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd" (OuterVolumeSpecName: "kube-api-access-zp7rd") pod "87e90838-8a40-4dd0-8b07-846f0a2e8aa0" (UID: "87e90838-8a40-4dd0-8b07-846f0a2e8aa0"). InnerVolumeSpecName "kube-api-access-zp7rd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:31:07.026056 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.026033    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util" (OuterVolumeSpecName: "util") pod "87e90838-8a40-4dd0-8b07-846f0a2e8aa0" (UID: "87e90838-8a40-4dd0-8b07-846f0a2e8aa0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:07.122640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.122616    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:07.122640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.122636    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zp7rd\" (UniqueName: \"kubernetes.io/projected/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-kube-api-access-zp7rd\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:07.122819 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.122646    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87e90838-8a40-4dd0-8b07-846f0a2e8aa0-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:07.769947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.769921    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67"
Sep 09 13:31:07.770093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.769921    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ncg67" event={"ID":"87e90838-8a40-4dd0-8b07-846f0a2e8aa0","Type":"ContainerDied","Data":"979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7"}
Sep 09 13:31:07.770093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:07.770034    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="979a3053464e066bdd7d90e0d6431495d4162d584f6bf0fae14607c468a877c7"
Sep 09 13:31:11.229501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229468    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"]
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229775    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="util"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229787    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="util"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229796    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="pull"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229803    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="pull"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229811    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="extract"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229817    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="extract"
Sep 09 13:31:11.229873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.229870    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="87e90838-8a40-4dd0-8b07-846f0a2e8aa0" containerName="extract"
Sep 09 13:31:11.269027 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.269000    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"]
Sep 09 13:31:11.269163 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.269086    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.272044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.272022    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 13:31:11.272439 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.272418    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 13:31:11.272612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.272500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-b4p28\""
Sep 09 13:31:11.457554 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.457515    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwvm6\" (UniqueName: \"kubernetes.io/projected/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-kube-api-access-fwvm6\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.457740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.457625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.558729 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.558671    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.558729 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.558715    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwvm6\" (UniqueName: \"kubernetes.io/projected/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-kube-api-access-fwvm6\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.559014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.558997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.570194 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.570167    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwvm6\" (UniqueName: \"kubernetes.io/projected/b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b-kube-api-access-fwvm6\") pod \"cert-manager-operator-controller-manager-55c444bf86-wlt24\" (UID: \"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.577999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.577981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"
Sep 09 13:31:11.894802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:11.894777    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24"]
Sep 09 13:31:11.896752 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:11.896728    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f7d3e0_f140_4d11_b1f9_7beb52ab4b4b.slice/crio-3abc64b6cab1017c10ef2dc65f2b24ccef66c312018093f74bb46a21369314b1 WatchSource:0}: Error finding container 3abc64b6cab1017c10ef2dc65f2b24ccef66c312018093f74bb46a21369314b1: Status 404 returned error can't find the container with id 3abc64b6cab1017c10ef2dc65f2b24ccef66c312018093f74bb46a21369314b1
Sep 09 13:31:12.790082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:12.790021    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24" event={"ID":"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b","Type":"ContainerStarted","Data":"3abc64b6cab1017c10ef2dc65f2b24ccef66c312018093f74bb46a21369314b1"}
Sep 09 13:31:13.796176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:13.796135    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24" event={"ID":"b0f7d3e0-f140-4d11-b1f9-7beb52ab4b4b","Type":"ContainerStarted","Data":"ff72ea6690800872c3590fa70afabafa8b8b4de2a97265162d34c1e8ad78b731"}
Sep 09 13:31:13.822219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:13.822132    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-wlt24" podStartSLOduration=1.169282055 podStartE2EDuration="2.822117366s" totalImagesPullingTime="1.652835311s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:31:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:31:13.820868567 +0000 UTC m=+292.709568554" watchObservedRunningTime="2026-09-09 13:31:13.822117366 +0000 UTC m=+292.710817356"
Sep 09 13:31:14.841506 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.841468    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"]
Sep 09 13:31:14.845056 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.845033    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.847625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.847603    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 13:31:14.847710 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.847672    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8bfrp\""
Sep 09 13:31:14.848891 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.848872    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 13:31:14.853784 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.853766    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"]
Sep 09 13:31:14.882745 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.882718    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.882827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.882758    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.882827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.882792    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbztg\" (UniqueName: \"kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.983475 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.983441    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.983596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.983488    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.983596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.983519    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbztg\" (UniqueName: \"kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.983884 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.983864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.983947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.983890    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:14.994641 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:14.994620    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbztg\" (UniqueName: \"kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:15.154878 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:15.154811    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:15.432394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:15.432371    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"]
Sep 09 13:31:15.434481 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:15.434451    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3511673_15e9_46c8_817e_d55705c5ec00.slice/crio-1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937 WatchSource:0}: Error finding container 1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937: Status 404 returned error can't find the container with id 1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937
Sep 09 13:31:15.805505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:15.805473    2559 generic.go:356] "Generic (PLEG): container finished" podID="a3511673-15e9-46c8-817e-d55705c5ec00" containerID="bde8c22ab6a296ecc464fc8007c89756bb2e7a33ad6ad3b413edac87ef23d765" exitCode=0
Sep 09 13:31:15.805654 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:15.805558    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx" event={"ID":"a3511673-15e9-46c8-817e-d55705c5ec00","Type":"ContainerDied","Data":"bde8c22ab6a296ecc464fc8007c89756bb2e7a33ad6ad3b413edac87ef23d765"}
Sep 09 13:31:15.805654 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:15.805620    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx" event={"ID":"a3511673-15e9-46c8-817e-d55705c5ec00","Type":"ContainerStarted","Data":"1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937"}
Sep 09 13:31:16.311632 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.311596    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54n4t"]
Sep 09 13:31:16.315284 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.315264    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.317869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.317855    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 13:31:16.318951 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.318936    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 13:31:16.318998 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.318976    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-gt2th\""
Sep 09 13:31:16.324788 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.324764    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54n4t"]
Sep 09 13:31:16.394407 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.394375    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7699j\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-kube-api-access-7699j\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.394552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.394429    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.495109 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.495072    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7699j\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-kube-api-access-7699j\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.495241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.495117    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.503721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.503689    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.503821 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.503768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7699j\" (UniqueName: \"kubernetes.io/projected/88b7d6fa-c161-4ad3-876b-032d2fb589f1-kube-api-access-7699j\") pod \"cert-manager-webhook-6486ffd997-54n4t\" (UID: \"88b7d6fa-c161-4ad3-876b-032d2fb589f1\") " pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.640357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.640260    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:16.908944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:16.908916    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-54n4t"]
Sep 09 13:31:16.911779 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:16.911743    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88b7d6fa_c161_4ad3_876b_032d2fb589f1.slice/crio-ed38da07c0ddb0ba317c8f2ad41e7914f90320da3e00b4df822d2eda23c8b989 WatchSource:0}: Error finding container ed38da07c0ddb0ba317c8f2ad41e7914f90320da3e00b4df822d2eda23c8b989: Status 404 returned error can't find the container with id ed38da07c0ddb0ba317c8f2ad41e7914f90320da3e00b4df822d2eda23c8b989
Sep 09 13:31:17.816393 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:17.816358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t" event={"ID":"88b7d6fa-c161-4ad3-876b-032d2fb589f1","Type":"ContainerStarted","Data":"ed38da07c0ddb0ba317c8f2ad41e7914f90320da3e00b4df822d2eda23c8b989"}
Sep 09 13:31:17.957895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:17.957857    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"]
Sep 09 13:31:17.969066 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:17.968949    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:17.970399 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:17.970371    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"]
Sep 09 13:31:17.971543 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:17.971518    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-s4lsv\""
Sep 09 13:31:18.009904 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.009872    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxlkk\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-kube-api-access-mxlkk\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.010065 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.010018    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.110999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.110927    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.110999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.110978    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxlkk\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-kube-api-access-mxlkk\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.119286 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.119253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.119399 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.119383    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxlkk\" (UniqueName: \"kubernetes.io/projected/3c179347-ddff-4dfe-83bc-55cea3c1b596-kube-api-access-mxlkk\") pod \"cert-manager-cainjector-559cdd8c87-2gx8v\" (UID: \"3c179347-ddff-4dfe-83bc-55cea3c1b596\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.280906 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.280879    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"
Sep 09 13:31:18.549630 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.549517    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v"]
Sep 09 13:31:18.551966 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:18.551942    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c179347_ddff_4dfe_83bc_55cea3c1b596.slice/crio-eea99d6dd83ddc7ec22ac103f46037ba09f5f5224ab6584e4400a1fc30092cb2 WatchSource:0}: Error finding container eea99d6dd83ddc7ec22ac103f46037ba09f5f5224ab6584e4400a1fc30092cb2: Status 404 returned error can't find the container with id eea99d6dd83ddc7ec22ac103f46037ba09f5f5224ab6584e4400a1fc30092cb2
Sep 09 13:31:18.821534 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.821457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v" event={"ID":"3c179347-ddff-4dfe-83bc-55cea3c1b596","Type":"ContainerStarted","Data":"eea99d6dd83ddc7ec22ac103f46037ba09f5f5224ab6584e4400a1fc30092cb2"}
Sep 09 13:31:18.823155 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.823132    2559 generic.go:356] "Generic (PLEG): container finished" podID="a3511673-15e9-46c8-817e-d55705c5ec00" containerID="5b4bbe16dfe7dfbc46db2959246bfe8890af3166c7c1f93f37468f366530e953" exitCode=0
Sep 09 13:31:18.823263 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:18.823183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx" event={"ID":"a3511673-15e9-46c8-817e-d55705c5ec00","Type":"ContainerDied","Data":"5b4bbe16dfe7dfbc46db2959246bfe8890af3166c7c1f93f37468f366530e953"}
Sep 09 13:31:19.830817 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:19.830779    2559 generic.go:356] "Generic (PLEG): container finished" podID="a3511673-15e9-46c8-817e-d55705c5ec00" containerID="1763250219a48b29dd3e4125bea3b823cc914d14aa899a0083059ed9f1c5accf" exitCode=0
Sep 09 13:31:19.830817 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:19.830827    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx" event={"ID":"a3511673-15e9-46c8-817e-d55705c5ec00","Type":"ContainerDied","Data":"1763250219a48b29dd3e4125bea3b823cc914d14aa899a0083059ed9f1c5accf"}
Sep 09 13:31:21.727014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.723586    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:21.736383 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.736363    2559 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 13:31:21.737671 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.737655    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle\") pod \"a3511673-15e9-46c8-817e-d55705c5ec00\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") "
Sep 09 13:31:21.737744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.737696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbztg\" (UniqueName: \"kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg\") pod \"a3511673-15e9-46c8-817e-d55705c5ec00\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") "
Sep 09 13:31:21.737744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.737718    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util\") pod \"a3511673-15e9-46c8-817e-d55705c5ec00\" (UID: \"a3511673-15e9-46c8-817e-d55705c5ec00\") "
Sep 09 13:31:21.738053 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.738021    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle" (OuterVolumeSpecName: "bundle") pod "a3511673-15e9-46c8-817e-d55705c5ec00" (UID: "a3511673-15e9-46c8-817e-d55705c5ec00"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:21.739639 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.739619    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg" (OuterVolumeSpecName: "kube-api-access-zbztg") pod "a3511673-15e9-46c8-817e-d55705c5ec00" (UID: "a3511673-15e9-46c8-817e-d55705c5ec00"). InnerVolumeSpecName "kube-api-access-zbztg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:31:21.742646 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.742624    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util" (OuterVolumeSpecName: "util") pod "a3511673-15e9-46c8-817e-d55705c5ec00" (UID: "a3511673-15e9-46c8-817e-d55705c5ec00"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:21.838171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.838153    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:21.838275 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.838172    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbztg\" (UniqueName: \"kubernetes.io/projected/a3511673-15e9-46c8-817e-d55705c5ec00-kube-api-access-zbztg\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:21.838275 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.838182    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a3511673-15e9-46c8-817e-d55705c5ec00-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:21.842720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.842689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx" event={"ID":"a3511673-15e9-46c8-817e-d55705c5ec00","Type":"ContainerDied","Data":"1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937"}
Sep 09 13:31:21.842844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.842728    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f5xhcx"
Sep 09 13:31:21.842844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.842743    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1dd9158aa1f69c96a9ed3fddef3248c0458598ccffff3af9f1b476cb4d4d9937"
Sep 09 13:31:21.844305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.844283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v" event={"ID":"3c179347-ddff-4dfe-83bc-55cea3c1b596","Type":"ContainerStarted","Data":"b1886fc6a432405309db790abf934916498fec628540b00952a232841f3a7afa"}
Sep 09 13:31:21.861242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:21.861203    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-2gx8v" podStartSLOduration=1.64197968 podStartE2EDuration="4.861190872s" totalImagesPullingTime="3.219211192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:31:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:31:21.860431627 +0000 UTC m=+300.749131619" watchObservedRunningTime="2026-09-09 13:31:21.861190872 +0000 UTC m=+300.749890861"
Sep 09 13:31:24.858342 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:24.858306    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t" event={"ID":"88b7d6fa-c161-4ad3-876b-032d2fb589f1","Type":"ContainerStarted","Data":"d3fdb2fd25ee35a358ab8537e1882816e511c02a6539b81b475aa2d163b60f3a"}
Sep 09 13:31:24.858720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:24.858362    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:24.882454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:24.882411    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t" podStartSLOduration=1.392674057 podStartE2EDuration="8.882397242s" totalImagesPullingTime="7.489723185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:31:24.881014756 +0000 UTC m=+303.769714758" watchObservedRunningTime="2026-09-09 13:31:24.882397242 +0000 UTC m=+303.771097231"
Sep 09 13:31:30.865526 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:30.865494    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-54n4t"
Sep 09 13:31:37.383044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383009    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"]
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383321    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="extract"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383333    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="extract"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383344    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="util"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383350    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="util"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383360    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="pull"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383365    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="pull"
Sep 09 13:31:37.383419 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.383412    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3511673-15e9-46c8-817e-d55705c5ec00" containerName="extract"
Sep 09 13:31:37.386622 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.386603    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.389506 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.389474    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 13:31:37.389637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.389593    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8bfrp\""
Sep 09 13:31:37.389707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.389677    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 13:31:37.394822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.394789    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"]
Sep 09 13:31:37.447287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.447264    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98c8f\" (UniqueName: \"kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.447405 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.447297    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.447405 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.447332    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.548349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.548321    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98c8f\" (UniqueName: \"kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.548509 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.548356    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.548509 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.548388    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.548731 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.548712    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.548783 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.548737    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.556196 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.556175    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98c8f\" (UniqueName: \"kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.695620 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.695555    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:37.966629 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:37.966586    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"]
Sep 09 13:31:37.968850 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:37.968824    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe249a42_2026_4796_915b_a19596876afd.slice/crio-83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1 WatchSource:0}: Error finding container 83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1: Status 404 returned error can't find the container with id 83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1
Sep 09 13:31:38.921742 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:38.921650    2559 generic.go:356] "Generic (PLEG): container finished" podID="fe249a42-2026-4796-915b-a19596876afd" containerID="1bc02d492cbc85e1df9959f89c73349f1993d04672718b8f54726b2443f68ad6" exitCode=0
Sep 09 13:31:38.921742 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:38.921725    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n" event={"ID":"fe249a42-2026-4796-915b-a19596876afd","Type":"ContainerDied","Data":"1bc02d492cbc85e1df9959f89c73349f1993d04672718b8f54726b2443f68ad6"}
Sep 09 13:31:38.922144 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:38.921777    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n" event={"ID":"fe249a42-2026-4796-915b-a19596876afd","Type":"ContainerStarted","Data":"83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1"}
Sep 09 13:31:38.922256 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:38.922240    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:31:40.931214 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:40.931180    2559 generic.go:356] "Generic (PLEG): container finished" podID="fe249a42-2026-4796-915b-a19596876afd" containerID="bad96ec47bce98cbb3093d1497f26721f7c7235444a6ba65608ad3fc42cc78f6" exitCode=0
Sep 09 13:31:40.931567 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:40.931266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n" event={"ID":"fe249a42-2026-4796-915b-a19596876afd","Type":"ContainerDied","Data":"bad96ec47bce98cbb3093d1497f26721f7c7235444a6ba65608ad3fc42cc78f6"}
Sep 09 13:31:41.937688 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:41.937655    2559 generic.go:356] "Generic (PLEG): container finished" podID="fe249a42-2026-4796-915b-a19596876afd" containerID="6d71019bc736cdf7e4cfebd29a78f1ecaaf100937e85d5fb06c785ba4ff95273" exitCode=0
Sep 09 13:31:41.938088 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:41.937717    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n" event={"ID":"fe249a42-2026-4796-915b-a19596876afd","Type":"ContainerDied","Data":"6d71019bc736cdf7e4cfebd29a78f1ecaaf100937e85d5fb06c785ba4ff95273"}
Sep 09 13:31:43.056967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.056943    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:43.193837 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.193766    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util\") pod \"fe249a42-2026-4796-915b-a19596876afd\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") "
Sep 09 13:31:43.193837 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.193831    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle\") pod \"fe249a42-2026-4796-915b-a19596876afd\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") "
Sep 09 13:31:43.194021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.193931    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-98c8f\" (UniqueName: \"kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f\") pod \"fe249a42-2026-4796-915b-a19596876afd\" (UID: \"fe249a42-2026-4796-915b-a19596876afd\") "
Sep 09 13:31:43.194454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.194428    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle" (OuterVolumeSpecName: "bundle") pod "fe249a42-2026-4796-915b-a19596876afd" (UID: "fe249a42-2026-4796-915b-a19596876afd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:43.195919 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.195896    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f" (OuterVolumeSpecName: "kube-api-access-98c8f") pod "fe249a42-2026-4796-915b-a19596876afd" (UID: "fe249a42-2026-4796-915b-a19596876afd"). InnerVolumeSpecName "kube-api-access-98c8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:31:43.278888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.278859    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util" (OuterVolumeSpecName: "util") pod "fe249a42-2026-4796-915b-a19596876afd" (UID: "fe249a42-2026-4796-915b-a19596876afd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:43.294424 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.294400    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-98c8f\" (UniqueName: \"kubernetes.io/projected/fe249a42-2026-4796-915b-a19596876afd-kube-api-access-98c8f\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:43.294424 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.294421    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:43.294536 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.294431    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fe249a42-2026-4796-915b-a19596876afd-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:43.948120 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.948049    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n" event={"ID":"fe249a42-2026-4796-915b-a19596876afd","Type":"ContainerDied","Data":"83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1"}
Sep 09 13:31:43.948120 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.948092    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="83a5109684913c6ef3bb00afd24e7db7425d845006c2d23075f6c1f3036611c1"
Sep 09 13:31:43.948120 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:43.948096    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af4l58n"
Sep 09 13:31:48.414738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.414709    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"]
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.414992    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="util"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415003    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="util"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415022    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="extract"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415028    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="extract"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415036    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="pull"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415041    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="pull"
Sep 09 13:31:48.415216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.415092    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe249a42-2026-4796-915b-a19596876afd" containerName="extract"
Sep 09 13:31:48.419436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.419419    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.422107 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.422089    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8bfrp\""
Sep 09 13:31:48.422197 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.422086    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 13:31:48.423230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.423217    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 13:31:48.428944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.428924    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"]
Sep 09 13:31:48.530997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.530970    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.531113 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.531030    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pjv9\" (UniqueName: \"kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.531113 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.531095    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.632017 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.631992    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.632114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.632042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pjv9\" (UniqueName: \"kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.632171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.632152    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.632366 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.632350    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.632451 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.632436    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.641148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.641126    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pjv9\" (UniqueName: \"kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:48.728178 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:48.728155    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:49.001070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:49.001045    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"]
Sep 09 13:31:49.003604 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:49.003557    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07ccc39b_7c02_435c_b017_0393c11fd510.slice/crio-ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6 WatchSource:0}: Error finding container ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6: Status 404 returned error can't find the container with id ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6
Sep 09 13:31:49.975315 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:49.975279    2559 generic.go:356] "Generic (PLEG): container finished" podID="07ccc39b-7c02-435c-b017-0393c11fd510" containerID="75bda4785f7b8a0310b9609f00de9e65f008ce609c713193835fd93e7937ed87" exitCode=0
Sep 09 13:31:49.975802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:49.975333    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd" event={"ID":"07ccc39b-7c02-435c-b017-0393c11fd510","Type":"ContainerDied","Data":"75bda4785f7b8a0310b9609f00de9e65f008ce609c713193835fd93e7937ed87"}
Sep 09 13:31:49.975802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:49.975396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd" event={"ID":"07ccc39b-7c02-435c-b017-0393c11fd510","Type":"ContainerStarted","Data":"ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6"}
Sep 09 13:31:50.383500 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.383430    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"]
Sep 09 13:31:50.387220 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.387204    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.390888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.390864    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 13:31:50.391045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.391013    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 13:31:50.391159 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.391059    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bk6bx\""
Sep 09 13:31:50.391224 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.391159    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 13:31:50.391508 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.391491    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 13:31:50.405270 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.405243    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"]
Sep 09 13:31:50.447347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.447320    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-tmp\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.447446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.447370    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.447446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.447390    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-manifests\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.447525 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.447447    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-webhook-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.447525 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.447493    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2z4z\" (UniqueName: \"kubernetes.io/projected/38df386b-935d-46ec-a35c-c213c0d9db09-kube-api-access-t2z4z\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548061 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548031    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-tmp\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548103    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-manifests\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548150    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-webhook-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548318 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548177    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2z4z\" (UniqueName: \"kubernetes.io/projected/38df386b-935d-46ec-a35c-c213c0d9db09-kube-api-access-t2z4z\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.548489 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.548465    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-manifests\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.550330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.550309    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38df386b-935d-46ec-a35c-c213c0d9db09-tmp\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.550448 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.550431    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.550534 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.550515    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38df386b-935d-46ec-a35c-c213c0d9db09-webhook-cert\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.556826 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.556803    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2z4z\" (UniqueName: \"kubernetes.io/projected/38df386b-935d-46ec-a35c-c213c0d9db09-kube-api-access-t2z4z\") pod \"opendatahub-operator-controller-manager-8469c9486d-pjp87\" (UID: \"38df386b-935d-46ec-a35c-c213c0d9db09\") " pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.701911 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.701661    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:50.982205 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.982168    2559 generic.go:356] "Generic (PLEG): container finished" podID="07ccc39b-7c02-435c-b017-0393c11fd510" containerID="866635d1957345ddfa72058186b69fb2d2726872015dc4c31f323f8fec8ab9fe" exitCode=0
Sep 09 13:31:50.982625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.982245    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd" event={"ID":"07ccc39b-7c02-435c-b017-0393c11fd510","Type":"ContainerDied","Data":"866635d1957345ddfa72058186b69fb2d2726872015dc4c31f323f8fec8ab9fe"}
Sep 09 13:31:50.991621 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:50.989231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"]
Sep 09 13:31:50.994009 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:31:50.993987    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38df386b_935d_46ec_a35c_c213c0d9db09.slice/crio-0b95b4579a7ec7360d612b6546526cbce15ba1178594029eeb5de43d9113a76d WatchSource:0}: Error finding container 0b95b4579a7ec7360d612b6546526cbce15ba1178594029eeb5de43d9113a76d: Status 404 returned error can't find the container with id 0b95b4579a7ec7360d612b6546526cbce15ba1178594029eeb5de43d9113a76d
Sep 09 13:31:51.991837 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:51.991801    2559 generic.go:356] "Generic (PLEG): container finished" podID="07ccc39b-7c02-435c-b017-0393c11fd510" containerID="a9a7fb8883d96d7167c65831f476389a6de55a9f43c73ec6149ce07ab346041f" exitCode=0
Sep 09 13:31:51.992278 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:51.991882    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd" event={"ID":"07ccc39b-7c02-435c-b017-0393c11fd510","Type":"ContainerDied","Data":"a9a7fb8883d96d7167c65831f476389a6de55a9f43c73ec6149ce07ab346041f"}
Sep 09 13:31:51.993590 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:51.993550    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87" event={"ID":"38df386b-935d-46ec-a35c-c213c0d9db09","Type":"ContainerStarted","Data":"0b95b4579a7ec7360d612b6546526cbce15ba1178594029eeb5de43d9113a76d"}
Sep 09 13:31:54.006537 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.006509    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd" event={"ID":"07ccc39b-7c02-435c-b017-0393c11fd510","Type":"ContainerDied","Data":"ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6"}
Sep 09 13:31:54.006856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.006556    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ae179615b2283f9a523cfa033c853e99baa7506aa2b791c1b01b97442618aad6"
Sep 09 13:31:54.014597 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.014561    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:54.079911 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.079885    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util\") pod \"07ccc39b-7c02-435c-b017-0393c11fd510\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") "
Sep 09 13:31:54.080001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.079937    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pjv9\" (UniqueName: \"kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9\") pod \"07ccc39b-7c02-435c-b017-0393c11fd510\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") "
Sep 09 13:31:54.080001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.079963    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle\") pod \"07ccc39b-7c02-435c-b017-0393c11fd510\" (UID: \"07ccc39b-7c02-435c-b017-0393c11fd510\") "
Sep 09 13:31:54.080882 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.080852    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle" (OuterVolumeSpecName: "bundle") pod "07ccc39b-7c02-435c-b017-0393c11fd510" (UID: "07ccc39b-7c02-435c-b017-0393c11fd510"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:54.082293 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.082268    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9" (OuterVolumeSpecName: "kube-api-access-2pjv9") pod "07ccc39b-7c02-435c-b017-0393c11fd510" (UID: "07ccc39b-7c02-435c-b017-0393c11fd510"). InnerVolumeSpecName "kube-api-access-2pjv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:31:54.086743 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.086718    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util" (OuterVolumeSpecName: "util") pod "07ccc39b-7c02-435c-b017-0393c11fd510" (UID: "07ccc39b-7c02-435c-b017-0393c11fd510"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:31:54.181561 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.181533    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:54.181561 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.181555    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pjv9\" (UniqueName: \"kubernetes.io/projected/07ccc39b-7c02-435c-b017-0393c11fd510-kube-api-access-2pjv9\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:54.181561 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:54.181564    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/07ccc39b-7c02-435c-b017-0393c11fd510-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:31:55.012738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:55.012708    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0nl5dd"
Sep 09 13:31:55.012738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:55.012722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87" event={"ID":"38df386b-935d-46ec-a35c-c213c0d9db09","Type":"ContainerStarted","Data":"d4d388b14a9cd7f663aff3ef1bacfe3757581cec4d2329e899c9e4c05115ea55"}
Sep 09 13:31:56.018520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:56.018482    2559 generic.go:356] "Generic (PLEG): container finished" podID="38df386b-935d-46ec-a35c-c213c0d9db09" containerID="d4d388b14a9cd7f663aff3ef1bacfe3757581cec4d2329e899c9e4c05115ea55" exitCode=0
Sep 09 13:31:56.018902 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:56.018562    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87" event={"ID":"38df386b-935d-46ec-a35c-c213c0d9db09","Type":"ContainerDied","Data":"d4d388b14a9cd7f663aff3ef1bacfe3757581cec4d2329e899c9e4c05115ea55"}
Sep 09 13:31:57.024752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:57.024710    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87" event={"ID":"38df386b-935d-46ec-a35c-c213c0d9db09","Type":"ContainerStarted","Data":"0ad342a3a75636b92c9f9d7fbf8a2ad14eebcd65bd65ce02b15efcf2835de533"}
Sep 09 13:31:57.025110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:57.024843    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:31:57.052584 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:31:57.052535    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87" podStartSLOduration=2.652977924 podStartE2EDuration="7.052521666s" totalImagesPullingTime="3.376623877s" totalInitContainerRuntime="1.022919865s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:31:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:31:57.049749044 +0000 UTC m=+335.938449033" watchObservedRunningTime="2026-09-09 13:31:57.052521666 +0000 UTC m=+335.941221655"
Sep 09 13:32:08.031917 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:08.031875    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8469c9486d-pjp87"
Sep 09 13:32:20.506568 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506491    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"]
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506810    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="extract"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506822    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="extract"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506835    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="pull"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506841    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="pull"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506850    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="util"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506856    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="util"
Sep 09 13:32:20.507026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.506905    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="07ccc39b-7c02-435c-b017-0393c11fd510" containerName="extract"
Sep 09 13:32:20.509733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.509718    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.512252 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.512220    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 13:32:20.513174 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.513154    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 13:32:20.513501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.513485    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 13:32:20.514250 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.514233    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-x7rbd\""
Sep 09 13:32:20.514298 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.514264    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 13:32:20.544716 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.544696    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"]
Sep 09 13:32:20.680698 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.680658    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6735ab87-3b5f-485d-8923-2e47396086ee-tls-certs\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.680870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.680748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6735ab87-3b5f-485d-8923-2e47396086ee-tmp\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.680870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.680796    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rq22\" (UniqueName: \"kubernetes.io/projected/6735ab87-3b5f-485d-8923-2e47396086ee-kube-api-access-4rq22\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.781830 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.781754    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6735ab87-3b5f-485d-8923-2e47396086ee-tmp\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.781830 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.781804    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rq22\" (UniqueName: \"kubernetes.io/projected/6735ab87-3b5f-485d-8923-2e47396086ee-kube-api-access-4rq22\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.781998 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.781857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6735ab87-3b5f-485d-8923-2e47396086ee-tls-certs\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.784172 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.784140    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6735ab87-3b5f-485d-8923-2e47396086ee-tmp\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.784409 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.784391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6735ab87-3b5f-485d-8923-2e47396086ee-tls-certs\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.804004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.803983    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rq22\" (UniqueName: \"kubernetes.io/projected/6735ab87-3b5f-485d-8923-2e47396086ee-kube-api-access-4rq22\") pod \"kube-auth-proxy-595c6d67d9-p8vtt\" (UID: \"6735ab87-3b5f-485d-8923-2e47396086ee\") " pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.818482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.818458    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"
Sep 09 13:32:20.915700 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.915663    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pslf5"]
Sep 09 13:32:20.919047 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.919026    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:20.922548 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.922526    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6vb5d\""
Sep 09 13:32:20.922656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.922551    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 09 13:32:20.936728 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:20.936705    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pslf5"]
Sep 09 13:32:21.085637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.085545    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4pfb\" (UniqueName: \"kubernetes.io/projected/a8ecf428-7915-466c-810d-9cfc32a081d7-kube-api-access-s4pfb\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.085768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.085678    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ecf428-7915-466c-810d-9cfc32a081d7-cert\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.091381 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.091358    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt"]
Sep 09 13:32:21.094051 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:21.094028    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6735ab87_3b5f_485d_8923_2e47396086ee.slice/crio-2afade1042675a64ae3f0a6f80ecddb27bffaee66f68db9ea20f91c951597f5e WatchSource:0}: Error finding container 2afade1042675a64ae3f0a6f80ecddb27bffaee66f68db9ea20f91c951597f5e: Status 404 returned error can't find the container with id 2afade1042675a64ae3f0a6f80ecddb27bffaee66f68db9ea20f91c951597f5e
Sep 09 13:32:21.134509 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.134480    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt" event={"ID":"6735ab87-3b5f-485d-8923-2e47396086ee","Type":"ContainerStarted","Data":"2afade1042675a64ae3f0a6f80ecddb27bffaee66f68db9ea20f91c951597f5e"}
Sep 09 13:32:21.186175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.186147    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4pfb\" (UniqueName: \"kubernetes.io/projected/a8ecf428-7915-466c-810d-9cfc32a081d7-kube-api-access-s4pfb\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.186283 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.186196    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ecf428-7915-466c-810d-9cfc32a081d7-cert\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.188488 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.188471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8ecf428-7915-466c-810d-9cfc32a081d7-cert\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.197924 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.197903    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4pfb\" (UniqueName: \"kubernetes.io/projected/a8ecf428-7915-466c-810d-9cfc32a081d7-kube-api-access-s4pfb\") pod \"odh-model-controller-7998c67dc4-pslf5\" (UID: \"a8ecf428-7915-466c-810d-9cfc32a081d7\") " pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.228808 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.228775    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:21.512963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.512940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pslf5"]
Sep 09 13:32:21.515384 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:21.515359    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8ecf428_7915_466c_810d_9cfc32a081d7.slice/crio-ce0c63f89a0e0b9dcf1362d0a8bbfd2625556f21cfaf94c08b5c79328e01dbaa WatchSource:0}: Error finding container ce0c63f89a0e0b9dcf1362d0a8bbfd2625556f21cfaf94c08b5c79328e01dbaa: Status 404 returned error can't find the container with id ce0c63f89a0e0b9dcf1362d0a8bbfd2625556f21cfaf94c08b5c79328e01dbaa
Sep 09 13:32:21.759681 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.759645    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4644v"]
Sep 09 13:32:21.763504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.763440    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:21.768708 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.768686    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 13:32:21.784195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.784131    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4644v"]
Sep 09 13:32:21.892356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.892319    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:21.892539 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.892365    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-725c7\" (UniqueName: \"kubernetes.io/projected/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-kube-api-access-725c7\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:21.992981 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.992922    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:21.992981 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:21.992967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-725c7\" (UniqueName: \"kubernetes.io/projected/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-kube-api-access-725c7\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:21.993508 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:21.993472    2559 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 09 13:32:21.993664 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:21.993541    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs podName:f43a4e83-5ca1-4670-bdcd-3daee12c5c17 nodeName:}" failed. No retries permitted until 2026-09-09 13:32:22.493521471 +0000 UTC m=+361.382221442 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs") pod "model-serving-api-54d89bd5fc-4644v" (UID: "f43a4e83-5ca1-4670-bdcd-3daee12c5c17") : secret "model-serving-api-tls" not found
Sep 09 13:32:22.008264 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.008231    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-725c7\" (UniqueName: \"kubernetes.io/projected/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-kube-api-access-725c7\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:22.025427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.025358    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:22.028712 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.028690    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.033927 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.033734    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 13:32:22.034037 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.034020    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 13:32:22.034395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.034212    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 13:32:22.034395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.034269    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 13:32:22.034552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.034500    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-qkfck\""
Sep 09 13:32:22.041643 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.039595    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:22.143235 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.143194    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5" event={"ID":"a8ecf428-7915-466c-810d-9cfc32a081d7","Type":"ContainerStarted","Data":"ce0c63f89a0e0b9dcf1362d0a8bbfd2625556f21cfaf94c08b5c79328e01dbaa"}
Sep 09 13:32:22.194816 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.194784    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.194841    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.194900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.194957    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bfsn\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195194 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.195059    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195194 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.195099    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.195194 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.195152    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.295915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.295833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.295915 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.295881    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296111 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.295924    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296111 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.295959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296111 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.296063    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296257 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.296160    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296257 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.296186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bfsn\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.296740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.296566    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.299078 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.298889    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.299970 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.299858    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.299970 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.299930    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.301041 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.301018    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.305554 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.305531    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bfsn\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.307143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.307108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-c8m6s\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.344112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.344087    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:22.499109 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.498693    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:22.504993 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.501680    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f43a4e83-5ca1-4670-bdcd-3daee12c5c17-tls-certs\") pod \"model-serving-api-54d89bd5fc-4644v\" (UID: \"f43a4e83-5ca1-4670-bdcd-3daee12c5c17\") " pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:22.641387 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.641206    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:22.677706 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:22.677481    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:23.415601 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:23.415529    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae0a546b_0cad_4221_b45e_8c1bdc7ef1d0.slice/crio-94a703b2e50c55a5be993fbe14dbde7843e998d98a31e5d1836100a9d8956fc6 WatchSource:0}: Error finding container 94a703b2e50c55a5be993fbe14dbde7843e998d98a31e5d1836100a9d8956fc6: Status 404 returned error can't find the container with id 94a703b2e50c55a5be993fbe14dbde7843e998d98a31e5d1836100a9d8956fc6
Sep 09 13:32:24.157192 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:24.157150    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" event={"ID":"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0","Type":"ContainerStarted","Data":"94a703b2e50c55a5be993fbe14dbde7843e998d98a31e5d1836100a9d8956fc6"}
Sep 09 13:32:25.756796 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:25.756772    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 13:32:26.014386 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.014359    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-4644v"]
Sep 09 13:32:26.171269 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.171233    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt" event={"ID":"6735ab87-3b5f-485d-8923-2e47396086ee","Type":"ContainerStarted","Data":"d1d061a5a03236a3a04624772f3ac022b543db658351b6fe1f48f2d6cdf35a72"}
Sep 09 13:32:26.173774 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.173745    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5" event={"ID":"a8ecf428-7915-466c-810d-9cfc32a081d7","Type":"ContainerStarted","Data":"93cbc67c2716afe8f591ad5827de6b2003b311777eb58abf6a402671c503a531"}
Sep 09 13:32:26.173944 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.173832    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:26.196024 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.195980    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-595c6d67d9-p8vtt" podStartSLOduration=1.5377092449999998 podStartE2EDuration="6.195962571s" totalImagesPullingTime="4.658253326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:26.194827696 +0000 UTC m=+365.083527713" watchObservedRunningTime="2026-09-09 13:32:26.195962571 +0000 UTC m=+365.084662561"
Sep 09 13:32:26.214860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:26.214800    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5" podStartSLOduration=1.968257353 podStartE2EDuration="6.214779344s" totalImagesPullingTime="4.246521991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:26.213062889 +0000 UTC m=+365.101762881" watchObservedRunningTime="2026-09-09 13:32:26.214779344 +0000 UTC m=+365.103479335"
Sep 09 13:32:27.038063 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:27.038031    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf43a4e83_5ca1_4670_bdcd_3daee12c5c17.slice/crio-c2cab4e5fd6986d4f423f8238064804e9ff596a46cce0d13edd04f8a2b9555f5 WatchSource:0}: Error finding container c2cab4e5fd6986d4f423f8238064804e9ff596a46cce0d13edd04f8a2b9555f5: Status 404 returned error can't find the container with id c2cab4e5fd6986d4f423f8238064804e9ff596a46cce0d13edd04f8a2b9555f5
Sep 09 13:32:27.110308 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.110269    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:32:27.110398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.110339    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:32:27.180617 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.180558    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" event={"ID":"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0","Type":"ContainerStarted","Data":"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"}
Sep 09 13:32:27.180786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.180764    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:27.181941 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.181914    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4644v" event={"ID":"f43a4e83-5ca1-4670-bdcd-3daee12c5c17","Type":"ContainerStarted","Data":"c2cab4e5fd6986d4f423f8238064804e9ff596a46cce0d13edd04f8a2b9555f5"}
Sep 09 13:32:27.206742 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:27.206699    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" podStartSLOduration=2.515173109 podStartE2EDuration="6.206682097s" totalImagesPullingTime="3.691508988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:27.205199677 +0000 UTC m=+366.093899669" watchObservedRunningTime="2026-09-09 13:32:27.206682097 +0000 UTC m=+366.095382087"
Sep 09 13:32:28.188510 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:28.188471    2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-c8m6s container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:32:28.188894 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:28.188536    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:32:29.192480 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:29.192448    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-4644v" event={"ID":"f43a4e83-5ca1-4670-bdcd-3daee12c5c17","Type":"ContainerStarted","Data":"aa919ee4336a538c025556d156301d4cecf075a71e290465bdfc0e66e23a92b4"}
Sep 09 13:32:29.192869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:29.192533    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:31.187890 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.187859    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:31.252104 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.252055    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-4644v" podStartSLOduration=8.643832217 podStartE2EDuration="10.252038633s" totalImagesPullingTime="1.608206416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:29.228047646 +0000 UTC m=+368.116747637" watchObservedRunningTime="2026-09-09 13:32:31.252038633 +0000 UTC m=+370.140738623"
Sep 09 13:32:31.832214 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.832172    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"]
Sep 09 13:32:31.836046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.836023    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.839088 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.839064    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 13:32:31.839212 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.839099    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-njx72\""
Sep 09 13:32:31.848795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.848776    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"]
Sep 09 13:32:31.851800 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.851784    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.854367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.854327    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jl7kq\""
Sep 09 13:32:31.854462 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.854427    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 13:32:31.857087 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.857044    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"]
Sep 09 13:32:31.868003 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.867981    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"]
Sep 09 13:32:31.882218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.882192    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4tz9\" (UniqueName: \"kubernetes.io/projected/7f26a138-c968-44fc-afe7-64480e465ba3-kube-api-access-z4tz9\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.882304 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.882233    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.882304 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.882287    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sz4q4\" (UniqueName: \"kubernetes.io/projected/9ed2da18-6939-455b-81c5-c05a0ed8e50f-kube-api-access-sz4q4\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.882379 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.882348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f26a138-c968-44fc-afe7-64480e465ba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.982946 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.982917    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4tz9\" (UniqueName: \"kubernetes.io/projected/7f26a138-c968-44fc-afe7-64480e465ba3-kube-api-access-z4tz9\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.982946 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.982949    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.983160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.982970    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sz4q4\" (UniqueName: \"kubernetes.io/projected/9ed2da18-6939-455b-81c5-c05a0ed8e50f-kube-api-access-sz4q4\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.983160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.983002    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f26a138-c968-44fc-afe7-64480e465ba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.983160 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:31.983085    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 13:32:31.983160 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:31.983153    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert podName:9ed2da18-6939-455b-81c5-c05a0ed8e50f nodeName:}" failed. No retries permitted until 2026-09-09 13:32:32.483134487 +0000 UTC m=+371.371834470 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert") pod "kserve-controller-manager-d6f4b65b8-6cpsw" (UID: "9ed2da18-6939-455b-81c5-c05a0ed8e50f") : secret "kserve-webhook-server-cert" not found
Sep 09 13:32:31.985546 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.985519    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f26a138-c968-44fc-afe7-64480e465ba3-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:31.994049 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.994024    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sz4q4\" (UniqueName: \"kubernetes.io/projected/9ed2da18-6939-455b-81c5-c05a0ed8e50f-kube-api-access-sz4q4\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:31.994304 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:31.994288    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4tz9\" (UniqueName: \"kubernetes.io/projected/7f26a138-c968-44fc-afe7-64480e465ba3-kube-api-access-z4tz9\") pod \"llmisvc-controller-manager-777c6bd7d5-q448v\" (UID: \"7f26a138-c968-44fc-afe7-64480e465ba3\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:32.163327 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.163236    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:32.282765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.282721    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:32.283150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.282934    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerName="discovery" containerID="cri-o://17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f" gracePeriod=30
Sep 09 13:32:32.444004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.443978    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"]
Sep 09 13:32:32.445998 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:32.445963    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7f26a138_c968_44fc_afe7_64480e465ba3.slice/crio-84bcef5cef5e54aeed111e942c17b6a3da1a7508b3009ea5e559453c53021a7e WatchSource:0}: Error finding container 84bcef5cef5e54aeed111e942c17b6a3da1a7508b3009ea5e559453c53021a7e: Status 404 returned error can't find the container with id 84bcef5cef5e54aeed111e942c17b6a3da1a7508b3009ea5e559453c53021a7e
Sep 09 13:32:32.489217 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.489190    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:32.489353 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:32.489294    2559 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 13:32:32.489411 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:32.489360    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert podName:9ed2da18-6939-455b-81c5-c05a0ed8e50f nodeName:}" failed. No retries permitted until 2026-09-09 13:32:33.48934416 +0000 UTC m=+372.378044128 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert") pod "kserve-controller-manager-d6f4b65b8-6cpsw" (UID: "9ed2da18-6939-455b-81c5-c05a0ed8e50f") : secret "kserve-webhook-server-cert" not found
Sep 09 13:32:32.513982 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.513962    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:32.589760 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589737    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.589900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589778    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.589900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589817    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bfsn\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.589900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589839    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.589900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589868    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.589900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589895    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.590155 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.589952    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts\") pod \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\" (UID: \"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0\") "
Sep 09 13:32:32.590425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.590340    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:32:32.592354 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592319    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs" (OuterVolumeSpecName: "local-certs") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:32:32.592450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592401    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:32:32.592450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592435    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:32:32.592528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592454    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:32:32.592528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592460    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts" (OuterVolumeSpecName: "cacerts") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:32:32.592737 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.592719    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn" (OuterVolumeSpecName: "kube-api-access-6bfsn") pod "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" (UID: "ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0"). InnerVolumeSpecName "kube-api-access-6bfsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:32:32.690866 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690841    2559 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-cacerts\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.690866 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690865    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-dns-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.690866 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690884    2559 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-kubeconfig\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.691141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690898    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bfsn\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-kube-api-access-6bfsn\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.691141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690923    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-token\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.691141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690936    2559 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-local-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:32.691141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:32.690953    2559 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0-istio-csr-ca-configmap\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:32:33.213341 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.213302    2559 generic.go:356] "Generic (PLEG): container finished" podID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerID="17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f" exitCode=0
Sep 09 13:32:33.213522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.213349    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"
Sep 09 13:32:33.213522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.213398    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" event={"ID":"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0","Type":"ContainerDied","Data":"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"}
Sep 09 13:32:33.213522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.213459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s" event={"ID":"ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0","Type":"ContainerDied","Data":"94a703b2e50c55a5be993fbe14dbde7843e998d98a31e5d1836100a9d8956fc6"}
Sep 09 13:32:33.213522 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.213486    2559 scope.go:117] "RemoveContainer" containerID="17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"
Sep 09 13:32:33.214883 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.214695    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v" event={"ID":"7f26a138-c968-44fc-afe7-64480e465ba3","Type":"ContainerStarted","Data":"84bcef5cef5e54aeed111e942c17b6a3da1a7508b3009ea5e559453c53021a7e"}
Sep 09 13:32:33.221886 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.221866    2559 scope.go:117] "RemoveContainer" containerID="17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"
Sep 09 13:32:33.222126 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:32:33.222110    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f\": container with ID starting with 17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f not found: ID does not exist" containerID="17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"
Sep 09 13:32:33.222177 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.222131    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f"} err="failed to get container status \"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f\": rpc error: code = NotFound desc = could not find container \"17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f\": container with ID starting with 17f5354fa9534d43f1c1fcec019b17d1bf9e7a0fde27f9d28d71b0ee99e1ec3f not found: ID does not exist"
Sep 09 13:32:33.272266 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.272212    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:33.293625 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.293598    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-c8m6s"]
Sep 09 13:32:33.497879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.497852    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:33.500095 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.500070    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ed2da18-6939-455b-81c5-c05a0ed8e50f-cert\") pod \"kserve-controller-manager-d6f4b65b8-6cpsw\" (UID: \"9ed2da18-6939-455b-81c5-c05a0ed8e50f\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:33.617214 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.617135    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" path="/var/lib/kubelet/pods/ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0/volumes"
Sep 09 13:32:33.646641 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:33.646615    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:34.145136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:34.145080    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"]
Sep 09 13:32:34.181240 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:34.181171    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ed2da18_6939_455b_81c5_c05a0ed8e50f.slice/crio-387a10f91b55c3dee09269918431986fbae3e67a9bd331b62ce5e1d10f7a7c34 WatchSource:0}: Error finding container 387a10f91b55c3dee09269918431986fbae3e67a9bd331b62ce5e1d10f7a7c34: Status 404 returned error can't find the container with id 387a10f91b55c3dee09269918431986fbae3e67a9bd331b62ce5e1d10f7a7c34
Sep 09 13:32:34.224634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:34.224595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw" event={"ID":"9ed2da18-6939-455b-81c5-c05a0ed8e50f","Type":"ContainerStarted","Data":"387a10f91b55c3dee09269918431986fbae3e67a9bd331b62ce5e1d10f7a7c34"}
Sep 09 13:32:36.239633 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:36.239592    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v" event={"ID":"7f26a138-c968-44fc-afe7-64480e465ba3","Type":"ContainerStarted","Data":"3de8dcb1ac5f199e1f2916e5b0d5bac345ae0d5fb9751f5c6a8e7d2f8629ca34"}
Sep 09 13:32:36.240095 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:36.239690    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:32:36.257929 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:36.257882    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v" podStartSLOduration=2.197600579 podStartE2EDuration="5.257870021s" totalImagesPullingTime="3.060269442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:36.25676739 +0000 UTC m=+375.145467382" watchObservedRunningTime="2026-09-09 13:32:36.257870021 +0000 UTC m=+375.146570011"
Sep 09 13:32:37.183810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:37.183783    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pslf5"
Sep 09 13:32:37.254240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:37.254200    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw" event={"ID":"9ed2da18-6939-455b-81c5-c05a0ed8e50f","Type":"ContainerStarted","Data":"9c72aa2b5f2757b09fb17ed0a6874a74e74653bbc1c98cd1cf580f62d11f488e"}
Sep 09 13:32:37.271833 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:37.271786    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw" podStartSLOduration=3.92781466 podStartE2EDuration="6.271770845s" totalImagesPullingTime="2.343956185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:37.270142784 +0000 UTC m=+376.158842781" watchObservedRunningTime="2026-09-09 13:32:37.271770845 +0000 UTC m=+376.160470835"
Sep 09 13:32:38.258481 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:38.258450    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:32:40.203372 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:40.203343    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-4644v"
Sep 09 13:32:55.529659 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.529630    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c97b554d8-7mbzm"]
Sep 09 13:32:55.530110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.529975    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerName="discovery"
Sep 09 13:32:55.530110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.529987    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerName="discovery"
Sep 09 13:32:55.530110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.530100    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae0a546b-0cad-4221-b45e-8c1bdc7ef1d0" containerName="discovery"
Sep 09 13:32:55.533357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.533339    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.547790 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.547769    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c97b554d8-7mbzm"]
Sep 09 13:32:55.685400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685369    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-trusted-ca-bundle\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685519 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685405    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685519 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685430    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wc8lg\" (UniqueName: \"kubernetes.io/projected/bda15c46-7d5d-4999-b924-8964ac7f5d2e-kube-api-access-wc8lg\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685519 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685487    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-oauth-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685518    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-service-ca\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685555    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-oauth-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.685662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.685594    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786376 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786305    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-trusted-ca-bundle\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786376 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786343    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786376 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786365    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wc8lg\" (UniqueName: \"kubernetes.io/projected/bda15c46-7d5d-4999-b924-8964ac7f5d2e-kube-api-access-wc8lg\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786431    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-oauth-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786470    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-service-ca\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786492    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-oauth-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.786626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.786515    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.787279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.787252    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.787279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.787270    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-trusted-ca-bundle\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.787457 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.787286    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-oauth-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.787457 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.787312    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bda15c46-7d5d-4999-b924-8964ac7f5d2e-service-ca\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.789074 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.789045    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-serving-cert\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.789171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.789071    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bda15c46-7d5d-4999-b924-8964ac7f5d2e-console-oauth-config\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.795990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.795967    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wc8lg\" (UniqueName: \"kubernetes.io/projected/bda15c46-7d5d-4999-b924-8964ac7f5d2e-kube-api-access-wc8lg\") pod \"console-6c97b554d8-7mbzm\" (UID: \"bda15c46-7d5d-4999-b924-8964ac7f5d2e\") " pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:55.842714 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:55.842696    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:32:56.115932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:56.115906    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c97b554d8-7mbzm"]
Sep 09 13:32:56.117298 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:32:56.117271    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbda15c46_7d5d_4999_b924_8964ac7f5d2e.slice/crio-0d29b294336f29b0d6a06febbfbd09f38cb83a5a871d41496e1ede98c6814e9c WatchSource:0}: Error finding container 0d29b294336f29b0d6a06febbfbd09f38cb83a5a871d41496e1ede98c6814e9c: Status 404 returned error can't find the container with id 0d29b294336f29b0d6a06febbfbd09f38cb83a5a871d41496e1ede98c6814e9c
Sep 09 13:32:56.342651 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:56.342615    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c97b554d8-7mbzm" event={"ID":"bda15c46-7d5d-4999-b924-8964ac7f5d2e","Type":"ContainerStarted","Data":"1a42d8355dfeb0fd1f2dc40ff7e2b019c51850461e40f02ed8ea47afbcbf300c"}
Sep 09 13:32:56.342794 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:56.342664    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c97b554d8-7mbzm" event={"ID":"bda15c46-7d5d-4999-b924-8964ac7f5d2e","Type":"ContainerStarted","Data":"0d29b294336f29b0d6a06febbfbd09f38cb83a5a871d41496e1ede98c6814e9c"}
Sep 09 13:32:56.363182 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:32:56.363119    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c97b554d8-7mbzm" podStartSLOduration=1.363092379 podStartE2EDuration="1.363092379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:32:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:32:56.360919378 +0000 UTC m=+395.249619389" watchObservedRunningTime="2026-09-09 13:32:56.363092379 +0000 UTC m=+395.251792373"
Sep 09 13:33:05.842861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:05.842828    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:33:05.843220 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:05.842888    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:33:05.847461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:05.847440    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:33:06.393467 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:06.393442    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c97b554d8-7mbzm"
Sep 09 13:33:06.470082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:06.470054    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:33:07.256858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:07.256820    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-q448v"
Sep 09 13:33:09.267628 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:09.267598    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-6cpsw"
Sep 09 13:33:27.857532 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.857503    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"]
Sep 09 13:33:27.862418 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.862400    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:27.871126 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.871108    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 13:33:27.871303 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.871291    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8bfrp\""
Sep 09 13:33:27.872351 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.872330    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 13:33:27.874711 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:27.874689    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"]
Sep 09 13:33:28.026515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.026485    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4drqd\" (UniqueName: \"kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.026672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.026522    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.026672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.026610    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.080766 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.080739    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"]
Sep 09 13:33:28.084356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.084336    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.098602 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.098561    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"]
Sep 09 13:33:28.128021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.127971    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.128021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.128024    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4drqd\" (UniqueName: \"kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.128155 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.128052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.128336 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.128322    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.128391 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.128375    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.141614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.141564    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4drqd\" (UniqueName: \"kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.150290 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.150270    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"]
Sep 09 13:33:28.154003 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.153988    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.163398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.163378    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"]
Sep 09 13:33:28.171198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.171180    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:28.229270 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.229231    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.229387 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.229345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkhd4\" (UniqueName: \"kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.229452 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.229404    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.237594 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.237548    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"]
Sep 09 13:33:28.241540 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.241523    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.254018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.253993    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"]
Sep 09 13:33:28.330605 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330557    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.330605 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330607    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.330852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.330852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330687    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.330852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330719    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkhd4\" (UniqueName: \"kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.330852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330740    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwkpp\" (UniqueName: \"kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.331002 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330926    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.331002 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.330960    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.342434 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.342412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkhd4\" (UniqueName: \"kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.392879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.392830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432116    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432190    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432218    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432262    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwkpp\" (UniqueName: \"kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432316    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.432428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432343    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28mnr\" (UniqueName: \"kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.432838 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432822    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.432996 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.432968    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.447062 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.447001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwkpp\" (UniqueName: \"kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.454324 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.454302    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"]
Sep 09 13:33:28.457710 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:28.457684    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e7f3ae_3548_454f_b5df_69db448eb13c.slice/crio-9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802 WatchSource:0}: Error finding container 9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802: Status 404 returned error can't find the container with id 9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802
Sep 09 13:33:28.462648 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.462630    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:28.492750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.492722    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79" event={"ID":"b1e7f3ae-3548-454f-b5df-69db448eb13c","Type":"ContainerStarted","Data":"9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802"}
Sep 09 13:33:28.533733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.533707    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.533840 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.533774    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.533900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.533853    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28mnr\" (UniqueName: \"kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.534063 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.534042    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.534119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.534100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.552430 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.552406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-28mnr\" (UniqueName: \"kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.552538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.552531    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:28.673990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.673960    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"]
Sep 09 13:33:28.675230 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:28.675210    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f71b0fe_9a04_48ab_a14c_f20ac49d7793.slice/crio-305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72 WatchSource:0}: Error finding container 305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72: Status 404 returned error can't find the container with id 305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72
Sep 09 13:33:28.745999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.745974    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"]
Sep 09 13:33:28.746932 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:28.746906    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ee1f770_91f8_4181_801d_18a0c76a2ada.slice/crio-9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282 WatchSource:0}: Error finding container 9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282: Status 404 returned error can't find the container with id 9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282
Sep 09 13:33:28.832973 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:28.832948    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"]
Sep 09 13:33:28.835632 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:28.835608    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod706119b0_78f8_453a_a342_3ba3826bcfb0.slice/crio-7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806 WatchSource:0}: Error finding container 7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806: Status 404 returned error can't find the container with id 7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806
Sep 09 13:33:29.500008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.499971    2559 generic.go:356] "Generic (PLEG): container finished" podID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerID="0022906bc8b83df9d084170b2b70f7bb7bfc0a4bfd216e000072ba9c288028d0" exitCode=0
Sep 09 13:33:29.500464 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.500062    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h" event={"ID":"706119b0-78f8-453a-a342-3ba3826bcfb0","Type":"ContainerDied","Data":"0022906bc8b83df9d084170b2b70f7bb7bfc0a4bfd216e000072ba9c288028d0"}
Sep 09 13:33:29.500464 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.500118    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h" event={"ID":"706119b0-78f8-453a-a342-3ba3826bcfb0","Type":"ContainerStarted","Data":"7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806"}
Sep 09 13:33:29.501672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.501618    2559 generic.go:356] "Generic (PLEG): container finished" podID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerID="31449313efa4ef6ac95c12b0ba313ae5c29939dadd2f5a388ed6a510c9d4ebc5" exitCode=0
Sep 09 13:33:29.501727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.501689    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45" event={"ID":"4ee1f770-91f8-4181-801d-18a0c76a2ada","Type":"ContainerDied","Data":"31449313efa4ef6ac95c12b0ba313ae5c29939dadd2f5a388ed6a510c9d4ebc5"}
Sep 09 13:33:29.501727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.501724    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45" event={"ID":"4ee1f770-91f8-4181-801d-18a0c76a2ada","Type":"ContainerStarted","Data":"9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282"}
Sep 09 13:33:29.503359 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.503341    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerID="c0fb3166d9c041e8b15f8b96022ec6240d68d4a18f700924db53124f99483dc3" exitCode=0
Sep 09 13:33:29.503452 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.503392    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9" event={"ID":"3f71b0fe-9a04-48ab-a14c-f20ac49d7793","Type":"ContainerDied","Data":"c0fb3166d9c041e8b15f8b96022ec6240d68d4a18f700924db53124f99483dc3"}
Sep 09 13:33:29.503452 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.503429    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9" event={"ID":"3f71b0fe-9a04-48ab-a14c-f20ac49d7793","Type":"ContainerStarted","Data":"305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72"}
Sep 09 13:33:29.505195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.505174    2559 generic.go:356] "Generic (PLEG): container finished" podID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerID="21f470654a2dd32d3c17c13dca989d19a029476fe31f7c1a01318c68c02e744c" exitCode=0
Sep 09 13:33:29.505267 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:29.505208    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79" event={"ID":"b1e7f3ae-3548-454f-b5df-69db448eb13c","Type":"ContainerDied","Data":"21f470654a2dd32d3c17c13dca989d19a029476fe31f7c1a01318c68c02e744c"}
Sep 09 13:33:31.490267 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.490233    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6d48f9c9c7-vv249" podUID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" containerName="console" containerID="cri-o://81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44" gracePeriod=15
Sep 09 13:33:31.517809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.517783    2559 generic.go:356] "Generic (PLEG): container finished" podID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerID="b2940117bbaf37cb46922fb5d5df76ace607d6b434fd859ef26cd1f221004aa9" exitCode=0
Sep 09 13:33:31.517900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.517863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79" event={"ID":"b1e7f3ae-3548-454f-b5df-69db448eb13c","Type":"ContainerDied","Data":"b2940117bbaf37cb46922fb5d5df76ace607d6b434fd859ef26cd1f221004aa9"}
Sep 09 13:33:31.519952 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.519932    2559 generic.go:356] "Generic (PLEG): container finished" podID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerID="b8ed02fb8ac04c3c69e9ed32f9d1cce73a3d4d08a08eba70bd82538a11179cbb" exitCode=0
Sep 09 13:33:31.520055 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.519997    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h" event={"ID":"706119b0-78f8-453a-a342-3ba3826bcfb0","Type":"ContainerDied","Data":"b8ed02fb8ac04c3c69e9ed32f9d1cce73a3d4d08a08eba70bd82538a11179cbb"}
Sep 09 13:33:31.521861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.521839    2559 generic.go:356] "Generic (PLEG): container finished" podID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerID="279cd22da0f3d6557b86595722042dc60bbfb7fc97037cf0408bc5d084f383cd" exitCode=0
Sep 09 13:33:31.521958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.521907    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45" event={"ID":"4ee1f770-91f8-4181-801d-18a0c76a2ada","Type":"ContainerDied","Data":"279cd22da0f3d6557b86595722042dc60bbfb7fc97037cf0408bc5d084f383cd"}
Sep 09 13:33:31.523986 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.523962    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerID="65335f20323ea1d1fc528e13158f65320ce40c779278263f219c9b8a64ffeba9" exitCode=0
Sep 09 13:33:31.524067 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.524041    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9" event={"ID":"3f71b0fe-9a04-48ab-a14c-f20ac49d7793","Type":"ContainerDied","Data":"65335f20323ea1d1fc528e13158f65320ce40c779278263f219c9b8a64ffeba9"}
Sep 09 13:33:31.912018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.911991    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:33:31.963918 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.963888    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.963938    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.963969    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964007    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hj7fw\" (UniqueName: \"kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964162 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964044    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964162 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964080    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:31.964375 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964348    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config" (OuterVolumeSpecName: "console-config") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:33:31.964451 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:33:31.964535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964510    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:33:31.964609 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.964555    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca" (OuterVolumeSpecName: "service-ca") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:33:31.965926 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.965907    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:33:31.966011 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:31.965993    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw" (OuterVolumeSpecName: "kube-api-access-hj7fw") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "kube-api-access-hj7fw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:33:32.064768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064700    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert\") pod \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\" (UID: \"3a97cbdc-03db-470c-8e25-ad7640a9a71c\") "
Sep 09 13:33:32.064888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064874    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hj7fw\" (UniqueName: \"kubernetes.io/projected/3a97cbdc-03db-470c-8e25-ad7640a9a71c-kube-api-access-hj7fw\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.064932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064890    2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-oauth-serving-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.064932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064899    2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.064932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064911    2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-oauth-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.064932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064923    2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-config\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.064932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.064932    2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3a97cbdc-03db-470c-8e25-ad7640a9a71c-trusted-ca-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.066418 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.066397    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3a97cbdc-03db-470c-8e25-ad7640a9a71c" (UID: "3a97cbdc-03db-470c-8e25-ad7640a9a71c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:33:32.165925 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.165901    2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3a97cbdc-03db-470c-8e25-ad7640a9a71c-console-serving-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:32.531342 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.531311    2559 generic.go:356] "Generic (PLEG): container finished" podID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerID="9e8142928fbdacdd27855637d8a52e7f118c244acb62ef7f951da038512046e2" exitCode=0
Sep 09 13:33:32.531733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.531394    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h" event={"ID":"706119b0-78f8-453a-a342-3ba3826bcfb0","Type":"ContainerDied","Data":"9e8142928fbdacdd27855637d8a52e7f118c244acb62ef7f951da038512046e2"}
Sep 09 13:33:32.533518 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.533494    2559 generic.go:356] "Generic (PLEG): container finished" podID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerID="09da7a4fa5248efdabb9485491ac6ccc5be0ceb7d83cf1a9eb29799b56e73330" exitCode=0
Sep 09 13:33:32.533623 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.533557    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45" event={"ID":"4ee1f770-91f8-4181-801d-18a0c76a2ada","Type":"ContainerDied","Data":"09da7a4fa5248efdabb9485491ac6ccc5be0ceb7d83cf1a9eb29799b56e73330"}
Sep 09 13:33:32.535386 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.535365    2559 generic.go:356] "Generic (PLEG): container finished" podID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerID="b81c5806b9e07588ff6fb7593def93d68a99d5ceee47ce6785671609e0f22058" exitCode=0
Sep 09 13:33:32.535473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.535434    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9" event={"ID":"3f71b0fe-9a04-48ab-a14c-f20ac49d7793","Type":"ContainerDied","Data":"b81c5806b9e07588ff6fb7593def93d68a99d5ceee47ce6785671609e0f22058"}
Sep 09 13:33:32.536728 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.536707    2559 generic.go:356] "Generic (PLEG): container finished" podID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" containerID="81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44" exitCode=0
Sep 09 13:33:32.536795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.536729    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d48f9c9c7-vv249" event={"ID":"3a97cbdc-03db-470c-8e25-ad7640a9a71c","Type":"ContainerDied","Data":"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"}
Sep 09 13:33:32.536795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.536749    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6d48f9c9c7-vv249"
Sep 09 13:33:32.536795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.536780    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6d48f9c9c7-vv249" event={"ID":"3a97cbdc-03db-470c-8e25-ad7640a9a71c","Type":"ContainerDied","Data":"e196af9b4e7b540bbad165541b2c7a26649a359a9b37fc3eb08529fa9e62c7f8"}
Sep 09 13:33:32.536917 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.536807    2559 scope.go:117] "RemoveContainer" containerID="81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"
Sep 09 13:33:32.539176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.539157    2559 generic.go:356] "Generic (PLEG): container finished" podID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerID="74c53535fed74308eb8e13f58d7cd0a08f394f9f072e168c6253e534ec7432c5" exitCode=0
Sep 09 13:33:32.539258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.539182    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79" event={"ID":"b1e7f3ae-3548-454f-b5df-69db448eb13c","Type":"ContainerDied","Data":"74c53535fed74308eb8e13f58d7cd0a08f394f9f072e168c6253e534ec7432c5"}
Sep 09 13:33:32.545722 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.545708    2559 scope.go:117] "RemoveContainer" containerID="81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"
Sep 09 13:33:32.545964 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:33:32.545949    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44\": container with ID starting with 81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44 not found: ID does not exist" containerID="81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"
Sep 09 13:33:32.546005 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.545970    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44"} err="failed to get container status \"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44\": rpc error: code = NotFound desc = could not find container \"81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44\": container with ID starting with 81cb1837b7e991280a2de40ba1a0c77b8d1e7176c30c820aa4ffca5afb63ac44 not found: ID does not exist"
Sep 09 13:33:32.623394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.623368    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:33:32.625842 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:32.625813    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6d48f9c9c7-vv249"]
Sep 09 13:33:33.618952 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.618914    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" path="/var/lib/kubelet/pods/3a97cbdc-03db-470c-8e25-ad7640a9a71c/volumes"
Sep 09 13:33:33.703206 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.703180    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:33.726690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.726627    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:33.738434 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.738414    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:33.740497 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.740481    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:33.777207 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777182    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle\") pod \"706119b0-78f8-453a-a342-3ba3826bcfb0\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") "
Sep 09 13:33:33.777330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777215    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util\") pod \"706119b0-78f8-453a-a342-3ba3826bcfb0\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") "
Sep 09 13:33:33.777330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777234    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkhd4\" (UniqueName: \"kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4\") pod \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") "
Sep 09 13:33:33.777330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777250    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util\") pod \"4ee1f770-91f8-4181-801d-18a0c76a2ada\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") "
Sep 09 13:33:33.777330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777278    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle\") pod \"4ee1f770-91f8-4181-801d-18a0c76a2ada\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777335    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util\") pod \"b1e7f3ae-3548-454f-b5df-69db448eb13c\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777383    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle\") pod \"b1e7f3ae-3548-454f-b5df-69db448eb13c\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777424    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwkpp\" (UniqueName: \"kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp\") pod \"4ee1f770-91f8-4181-801d-18a0c76a2ada\" (UID: \"4ee1f770-91f8-4181-801d-18a0c76a2ada\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777454    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle\") pod \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777478    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util\") pod \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\" (UID: \"3f71b0fe-9a04-48ab-a14c-f20ac49d7793\") "
Sep 09 13:33:33.777533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.777517    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4drqd\" (UniqueName: \"kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd\") pod \"b1e7f3ae-3548-454f-b5df-69db448eb13c\" (UID: \"b1e7f3ae-3548-454f-b5df-69db448eb13c\") "
Sep 09 13:33:33.780566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.778266    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle" (OuterVolumeSpecName: "bundle") pod "706119b0-78f8-453a-a342-3ba3826bcfb0" (UID: "706119b0-78f8-453a-a342-3ba3826bcfb0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.780566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.778396    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle" (OuterVolumeSpecName: "bundle") pod "4ee1f770-91f8-4181-801d-18a0c76a2ada" (UID: "4ee1f770-91f8-4181-801d-18a0c76a2ada"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.780566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.779708    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle" (OuterVolumeSpecName: "bundle") pod "b1e7f3ae-3548-454f-b5df-69db448eb13c" (UID: "b1e7f3ae-3548-454f-b5df-69db448eb13c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.780566 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.780522    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle" (OuterVolumeSpecName: "bundle") pod "3f71b0fe-9a04-48ab-a14c-f20ac49d7793" (UID: "3f71b0fe-9a04-48ab-a14c-f20ac49d7793"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.781528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.781498    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4" (OuterVolumeSpecName: "kube-api-access-mkhd4") pod "3f71b0fe-9a04-48ab-a14c-f20ac49d7793" (UID: "3f71b0fe-9a04-48ab-a14c-f20ac49d7793"). InnerVolumeSpecName "kube-api-access-mkhd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:33:33.781814 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.781784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd" (OuterVolumeSpecName: "kube-api-access-4drqd") pod "b1e7f3ae-3548-454f-b5df-69db448eb13c" (UID: "b1e7f3ae-3548-454f-b5df-69db448eb13c"). InnerVolumeSpecName "kube-api-access-4drqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:33:33.782032 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.782003    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp" (OuterVolumeSpecName: "kube-api-access-rwkpp") pod "4ee1f770-91f8-4181-801d-18a0c76a2ada" (UID: "4ee1f770-91f8-4181-801d-18a0c76a2ada"). InnerVolumeSpecName "kube-api-access-rwkpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:33:33.785319 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.785295    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util" (OuterVolumeSpecName: "util") pod "b1e7f3ae-3548-454f-b5df-69db448eb13c" (UID: "b1e7f3ae-3548-454f-b5df-69db448eb13c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.785395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.785361    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util" (OuterVolumeSpecName: "util") pod "4ee1f770-91f8-4181-801d-18a0c76a2ada" (UID: "4ee1f770-91f8-4181-801d-18a0c76a2ada"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.785927 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.785902    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util" (OuterVolumeSpecName: "util") pod "706119b0-78f8-453a-a342-3ba3826bcfb0" (UID: "706119b0-78f8-453a-a342-3ba3826bcfb0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.787854 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.787829    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util" (OuterVolumeSpecName: "util") pod "3f71b0fe-9a04-48ab-a14c-f20ac49d7793" (UID: "3f71b0fe-9a04-48ab-a14c-f20ac49d7793"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:33:33.878994 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.878929    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-28mnr\" (UniqueName: \"kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr\") pod \"706119b0-78f8-453a-a342-3ba3826bcfb0\" (UID: \"706119b0-78f8-453a-a342-3ba3826bcfb0\") "
Sep 09 13:33:33.879174 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879157    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4drqd\" (UniqueName: \"kubernetes.io/projected/b1e7f3ae-3548-454f-b5df-69db448eb13c-kube-api-access-4drqd\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879180    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879196    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/706119b0-78f8-453a-a342-3ba3826bcfb0-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879209    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkhd4\" (UniqueName: \"kubernetes.io/projected/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-kube-api-access-mkhd4\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879224    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879236    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4ee1f770-91f8-4181-801d-18a0c76a2ada-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879247    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879259    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b1e7f3ae-3548-454f-b5df-69db448eb13c-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879270    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwkpp\" (UniqueName: \"kubernetes.io/projected/4ee1f770-91f8-4181-801d-18a0c76a2ada-kube-api-access-rwkpp\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879282    2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-bundle\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.879306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.879293    2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f71b0fe-9a04-48ab-a14c-f20ac49d7793-util\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:33.880857 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.880838    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr" (OuterVolumeSpecName: "kube-api-access-28mnr") pod "706119b0-78f8-453a-a342-3ba3826bcfb0" (UID: "706119b0-78f8-453a-a342-3ba3826bcfb0"). InnerVolumeSpecName "kube-api-access-28mnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:33:33.980655 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:33.980633    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-28mnr\" (UniqueName: \"kubernetes.io/projected/706119b0-78f8-453a-a342-3ba3826bcfb0-kube-api-access-28mnr\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:33:34.552973 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.552933    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79" event={"ID":"b1e7f3ae-3548-454f-b5df-69db448eb13c","Type":"ContainerDied","Data":"9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802"}
Sep 09 13:33:34.553167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.553010    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9432f87a800d45086cf5c6730fe1709d409e61f4b51d2541e2754d8b005c4802"
Sep 09 13:33:34.553167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.552942    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6wjp79"
Sep 09 13:33:34.555025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.554962    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h"
Sep 09 13:33:34.555025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.554977    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cnhn4h" event={"ID":"706119b0-78f8-453a-a342-3ba3826bcfb0","Type":"ContainerDied","Data":"7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806"}
Sep 09 13:33:34.555025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.555023    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7f3d9c5674eec5698188928c765b1e39e68bee31f98879490684a712ab333806"
Sep 09 13:33:34.557675 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.557648    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45" event={"ID":"4ee1f770-91f8-4181-801d-18a0c76a2ada","Type":"ContainerDied","Data":"9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282"}
Sep 09 13:33:34.557675 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.557672    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d75zr45"
Sep 09 13:33:34.557868 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.557691    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9250823bafdf7ebb947d2465a7059f2354dd457c246b77abd0587c1633a5d282"
Sep 09 13:33:34.559937 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.559910    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9"
Sep 09 13:33:34.559937 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.559918    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a278r9" event={"ID":"3f71b0fe-9a04-48ab-a14c-f20ac49d7793","Type":"ContainerDied","Data":"305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72"}
Sep 09 13:33:34.560102 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:34.559954    2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="305ba68eade56a8e00f678e27e6d26d65c96abd081c4dfc5eca37396daf60a72"
Sep 09 13:33:45.013660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.013566    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-76fd5"]
Sep 09 13:33:45.013995 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.013979    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="extract"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.013997    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="extract"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014013    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="util"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014018    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="util"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014025    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="pull"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014031    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="pull"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014041    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="util"
Sep 09 13:33:45.014050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014046    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="util"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014053    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" containerName="console"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014058    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" containerName="console"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014063    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014068    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014074    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014079    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014084    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014089    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="pull"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014093    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014098    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014105    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="util"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014109    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="util"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014116    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="util"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014121    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="util"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014126    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014130    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014138    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014143    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014193    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="706119b0-78f8-453a-a342-3ba3826bcfb0" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014201    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f71b0fe-9a04-48ab-a14c-f20ac49d7793" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014208    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1e7f3ae-3548-454f-b5df-69db448eb13c" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014213    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ee1f770-91f8-4181-801d-18a0c76a2ada" containerName="extract"
Sep 09 13:33:45.014242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.014219    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a97cbdc-03db-470c-8e25-ad7640a9a71c" containerName="console"
Sep 09 13:33:45.018420 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.018401    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:45.023176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.023154    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 13:33:45.023259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.023207    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-jd9jt\""
Sep 09 13:33:45.023259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.023154    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 13:33:45.040567 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.040542    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-76fd5"]
Sep 09 13:33:45.072963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.072936    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbswv\" (UniqueName: \"kubernetes.io/projected/d97d04ee-24b0-4ed1-975d-54c05dd219a9-kube-api-access-jbswv\") pod \"authorino-operator-76b478bd78-76fd5\" (UID: \"d97d04ee-24b0-4ed1-975d-54c05dd219a9\") " pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:45.174222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.174195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbswv\" (UniqueName: \"kubernetes.io/projected/d97d04ee-24b0-4ed1-975d-54c05dd219a9-kube-api-access-jbswv\") pod \"authorino-operator-76b478bd78-76fd5\" (UID: \"d97d04ee-24b0-4ed1-975d-54c05dd219a9\") " pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:45.187200 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.187174    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbswv\" (UniqueName: \"kubernetes.io/projected/d97d04ee-24b0-4ed1-975d-54c05dd219a9-kube-api-access-jbswv\") pod \"authorino-operator-76b478bd78-76fd5\" (UID: \"d97d04ee-24b0-4ed1-975d-54c05dd219a9\") " pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:45.328499 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.328442    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:45.621036 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.621016    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-76fd5"]
Sep 09 13:33:45.621699 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:45.621674    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd97d04ee_24b0_4ed1_975d_54c05dd219a9.slice/crio-02850c069e88a173564443a0cc8f6e622c18171913f98ffb297f86dcc2333893 WatchSource:0}: Error finding container 02850c069e88a173564443a0cc8f6e622c18171913f98ffb297f86dcc2333893: Status 404 returned error can't find the container with id 02850c069e88a173564443a0cc8f6e622c18171913f98ffb297f86dcc2333893
Sep 09 13:33:45.633280 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:45.633252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5" event={"ID":"d97d04ee-24b0-4ed1-975d-54c05dd219a9","Type":"ContainerStarted","Data":"02850c069e88a173564443a0cc8f6e622c18171913f98ffb297f86dcc2333893"}
Sep 09 13:33:48.019370 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.019330    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"]
Sep 09 13:33:48.023912 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.023889    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:48.027594 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.027554    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t4gv2\""
Sep 09 13:33:48.039878 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.039853    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"]
Sep 09 13:33:48.096672 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.096640    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtb4z\" (UniqueName: \"kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z\") pod \"limitador-operator-controller-manager-74495c69f8-l8ts5\" (UID: \"14377021-bc79-4103-bdd9-662aef02cc4f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:48.198048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.198006    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtb4z\" (UniqueName: \"kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z\") pod \"limitador-operator-controller-manager-74495c69f8-l8ts5\" (UID: \"14377021-bc79-4103-bdd9-662aef02cc4f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:48.207505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.207478    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtb4z\" (UniqueName: \"kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z\") pod \"limitador-operator-controller-manager-74495c69f8-l8ts5\" (UID: \"14377021-bc79-4103-bdd9-662aef02cc4f\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:48.335880 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.335822    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:48.630281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.630256    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"]
Sep 09 13:33:48.632159 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:48.632134    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14377021_bc79_4103_bdd9_662aef02cc4f.slice/crio-3583204ba4c0fd3025280c6d4e4802507a1f16fb381bc0a3af736841409f194a WatchSource:0}: Error finding container 3583204ba4c0fd3025280c6d4e4802507a1f16fb381bc0a3af736841409f194a: Status 404 returned error can't find the container with id 3583204ba4c0fd3025280c6d4e4802507a1f16fb381bc0a3af736841409f194a
Sep 09 13:33:48.651317 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:48.651290    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5" event={"ID":"14377021-bc79-4103-bdd9-662aef02cc4f","Type":"ContainerStarted","Data":"3583204ba4c0fd3025280c6d4e4802507a1f16fb381bc0a3af736841409f194a"}
Sep 09 13:33:51.671279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:51.671192    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5" event={"ID":"14377021-bc79-4103-bdd9-662aef02cc4f","Type":"ContainerStarted","Data":"8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d"}
Sep 09 13:33:51.671688 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:51.671346    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:33:51.692014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:51.691973    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5" podStartSLOduration=2.095142577 podStartE2EDuration="4.691960228s" totalImagesPullingTime="2.596817651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:33:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:33:51.690000458 +0000 UTC m=+450.578700449" watchObservedRunningTime="2026-09-09 13:33:51.691960228 +0000 UTC m=+450.580660219"
Sep 09 13:33:54.090932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.090896    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"]
Sep 09 13:33:54.094545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.094528    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.097123 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.097101    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g67j8\""
Sep 09 13:33:54.108162 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.108136    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"]
Sep 09 13:33:54.149495 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.149465    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxpbj\" (UniqueName: \"kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.149608 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.149501    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.250248 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.250221    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxpbj\" (UniqueName: \"kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.250357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.250261    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.250662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.250639    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.261398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.261372    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxpbj\" (UniqueName: \"kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.404639 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.404551    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:54.702093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:54.702063    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"]
Sep 09 13:33:54.840532 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:54.840487    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb09a7f6e_0c80_498e_add7_740c3a8ca998.slice/crio-51b76ac42158cc3f2c77a5f5bd94f42f6c3cda24a910f5cb11745dd252ad01fa WatchSource:0}: Error finding container 51b76ac42158cc3f2c77a5f5bd94f42f6c3cda24a910f5cb11745dd252ad01fa: Status 404 returned error can't find the container with id 51b76ac42158cc3f2c77a5f5bd94f42f6c3cda24a910f5cb11745dd252ad01fa
Sep 09 13:33:55.697626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:55.697568    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5" event={"ID":"d97d04ee-24b0-4ed1-975d-54c05dd219a9","Type":"ContainerStarted","Data":"f1866c8526ff8b4f10bea7893a4d0147109078b864574d4c68b45711716c7eec"}
Sep 09 13:33:55.698069 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:55.697765    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:33:55.699267 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:55.699244    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv" event={"ID":"b09a7f6e-0c80-498e-add7-740c3a8ca998","Type":"ContainerStarted","Data":"51b76ac42158cc3f2c77a5f5bd94f42f6c3cda24a910f5cb11745dd252ad01fa"}
Sep 09 13:33:55.717612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:55.717554    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5" podStartSLOduration=2.435658924 podStartE2EDuration="11.71754051s" totalImagesPullingTime="9.281881586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:33:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:33:55.715840025 +0000 UTC m=+454.604540017" watchObservedRunningTime="2026-09-09 13:33:55.71754051 +0000 UTC m=+454.606240499"
Sep 09 13:33:56.984307 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:56.984273    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"]
Sep 09 13:33:56.987941 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:56.987920    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:33:56.992811 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:56.992786    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 13:33:56.992930 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:56.992858    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-47jf8\""
Sep 09 13:33:57.002775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.002752    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"]
Sep 09 13:33:57.074221 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.074190    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6xd2\" (UniqueName: \"kubernetes.io/projected/ec21a6e4-4aa8-46bc-9540-36b3b2d993a9-kube-api-access-q6xd2\") pod \"dns-operator-controller-manager-7b84cd7c9c-qprnj\" (UID: \"ec21a6e4-4aa8-46bc-9540-36b3b2d993a9\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:33:57.175285 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.175249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q6xd2\" (UniqueName: \"kubernetes.io/projected/ec21a6e4-4aa8-46bc-9540-36b3b2d993a9-kube-api-access-q6xd2\") pod \"dns-operator-controller-manager-7b84cd7c9c-qprnj\" (UID: \"ec21a6e4-4aa8-46bc-9540-36b3b2d993a9\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:33:57.185008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.184986    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6xd2\" (UniqueName: \"kubernetes.io/projected/ec21a6e4-4aa8-46bc-9540-36b3b2d993a9-kube-api-access-q6xd2\") pod \"dns-operator-controller-manager-7b84cd7c9c-qprnj\" (UID: \"ec21a6e4-4aa8-46bc-9540-36b3b2d993a9\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:33:57.301079 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.300981    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:33:57.586129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:57.586103    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"]
Sep 09 13:33:58.594064 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:33:58.594027    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec21a6e4_4aa8_46bc_9540_36b3b2d993a9.slice/crio-d0296c247f08703d329edb8e902b8b29db8449c719183a56aacbecd5e279af47 WatchSource:0}: Error finding container d0296c247f08703d329edb8e902b8b29db8449c719183a56aacbecd5e279af47: Status 404 returned error can't find the container with id d0296c247f08703d329edb8e902b8b29db8449c719183a56aacbecd5e279af47
Sep 09 13:33:58.718458 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:58.718430    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj" event={"ID":"ec21a6e4-4aa8-46bc-9540-36b3b2d993a9","Type":"ContainerStarted","Data":"d0296c247f08703d329edb8e902b8b29db8449c719183a56aacbecd5e279af47"}
Sep 09 13:33:59.727962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:59.727924    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv" event={"ID":"b09a7f6e-0c80-498e-add7-740c3a8ca998","Type":"ContainerStarted","Data":"3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2"}
Sep 09 13:33:59.728514 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:59.728060    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:33:59.751942 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:33:59.751885    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv" podStartSLOduration=1.933759887 podStartE2EDuration="5.751867681s" totalImagesPullingTime="3.818107794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:33:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:33:59.749826668 +0000 UTC m=+458.638526671" watchObservedRunningTime="2026-09-09 13:33:59.751867681 +0000 UTC m=+458.640567673"
Sep 09 13:34:00.735174 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:00.735129    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj" event={"ID":"ec21a6e4-4aa8-46bc-9540-36b3b2d993a9","Type":"ContainerStarted","Data":"2a2334a52fe3ce57cc12404d6d8853939e7d52fd03ab087393faf26bd375022c"}
Sep 09 13:34:00.735533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:00.735201    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:34:00.756176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:00.756124    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj" podStartSLOduration=3.064442575 podStartE2EDuration="4.756105293s" totalImagesPullingTime="1.691662718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:00.755435774 +0000 UTC m=+459.644135764" watchObservedRunningTime="2026-09-09 13:34:00.756105293 +0000 UTC m=+459.644805284"
Sep 09 13:34:01.581443 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.581410    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"]
Sep 09 13:34:01.585136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.585121    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.591358 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.591319    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-5fch6\""
Sep 09 13:34:01.591515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.591401    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 09 13:34:01.591515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.591484    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 09 13:34:01.599982 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.599961    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"]
Sep 09 13:34:01.712928 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.712889    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h59x\" (UniqueName: \"kubernetes.io/projected/dd9dac10-198a-4aba-9909-e056c7e61546-kube-api-access-5h59x\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.712928 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.712924    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9dac10-198a-4aba-9909-e056c7e61546-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.713148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.712994    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9dac10-198a-4aba-9909-e056c7e61546-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.814358 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.814329    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9dac10-198a-4aba-9909-e056c7e61546-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.814959 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.814381    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h59x\" (UniqueName: \"kubernetes.io/projected/dd9dac10-198a-4aba-9909-e056c7e61546-kube-api-access-5h59x\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.814959 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.814399    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9dac10-198a-4aba-9909-e056c7e61546-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.815240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.815215    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/dd9dac10-198a-4aba-9909-e056c7e61546-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.816756 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.816735    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/dd9dac10-198a-4aba-9909-e056c7e61546-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.822556 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.822532    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h59x\" (UniqueName: \"kubernetes.io/projected/dd9dac10-198a-4aba-9909-e056c7e61546-kube-api-access-5h59x\") pod \"kuadrant-console-plugin-fbdd4f445-fs2bh\" (UID: \"dd9dac10-198a-4aba-9909-e056c7e61546\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:01.894394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:01.894340    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"
Sep 09 13:34:02.163799 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:02.163766    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh"]
Sep 09 13:34:02.165500 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:02.165471    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd9dac10_198a_4aba_9909_e056c7e61546.slice/crio-5c95596c6aa4798d80c95180ee02e3f1190faa4126cd48fcf7ca9f7bfbaaf0bf WatchSource:0}: Error finding container 5c95596c6aa4798d80c95180ee02e3f1190faa4126cd48fcf7ca9f7bfbaaf0bf: Status 404 returned error can't find the container with id 5c95596c6aa4798d80c95180ee02e3f1190faa4126cd48fcf7ca9f7bfbaaf0bf
Sep 09 13:34:02.679596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:02.679553    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:02.747635 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:02.747594    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh" event={"ID":"dd9dac10-198a-4aba-9909-e056c7e61546","Type":"ContainerStarted","Data":"5c95596c6aa4798d80c95180ee02e3f1190faa4126cd48fcf7ca9f7bfbaaf0bf"}
Sep 09 13:34:06.709059 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:06.709020    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-76fd5"
Sep 09 13:34:07.785421 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:07.785386    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh" event={"ID":"dd9dac10-198a-4aba-9909-e056c7e61546","Type":"ContainerStarted","Data":"4c0b02f77d30565cd9fff191f7345ac5bdd3cdd63e660f80f99650906ce596fd"}
Sep 09 13:34:07.817964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:07.817920    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-fs2bh" podStartSLOduration=2.000768959 podStartE2EDuration="6.817905658s" totalImagesPullingTime="4.817136699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:07.817642139 +0000 UTC m=+466.706342130" watchObservedRunningTime="2026-09-09 13:34:07.817905658 +0000 UTC m=+466.706605648"
Sep 09 13:34:10.737499 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:10.737464    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:11.743152 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:11.743121    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-qprnj"
Sep 09 13:34:13.349387 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.349353    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"]
Sep 09 13:34:13.350199 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.350161    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" containerName="manager" containerID="cri-o://3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2" gracePeriod=2
Sep 09 13:34:13.353807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.353772    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"]
Sep 09 13:34:13.364613 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.364538    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"]
Sep 09 13:34:13.364723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.364715    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"]
Sep 09 13:34:13.365205 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.365173    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" containerName="manager" containerID="cri-o://8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d" gracePeriod=2
Sep 09 13:34:13.387587 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.387544    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.390491 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.390451    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-l8ts5\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:13.398140 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.398114    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.490769 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.490736    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:13.491345 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491320    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" containerName="manager"
Sep 09 13:34:13.491345 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491347    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" containerName="manager"
Sep 09 13:34:13.491493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491358    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" containerName="manager"
Sep 09 13:34:13.491493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491367    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" containerName="manager"
Sep 09 13:34:13.491493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491455    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" containerName="manager"
Sep 09 13:34:13.491493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.491467    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" containerName="manager"
Sep 09 13:34:13.494892 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.494873    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.522798 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.522770    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"]
Sep 09 13:34:13.526544 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.526523    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:13.558365 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.558336    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:13.602232 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.602142    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-l8ts5\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:13.607599 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.607544    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.610232 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.610204    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bwwrv\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.613245 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.613205    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-l8ts5\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:13.613510 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.613492    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwlhg\" (UniqueName: \"kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.613593 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.613544    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.613659 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.613600    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jxcz\" (UniqueName: \"kubernetes.io/projected/22dc8db1-1151-42d3-b76a-4846747bcd9b-kube-api-access-6jxcz\") pod \"limitador-operator-controller-manager-74495c69f8-pqqzl\" (UID: \"22dc8db1-1151-42d3-b76a-4846747bcd9b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:13.624519 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.624495    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"]
Sep 09 13:34:13.633008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.632986    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.635760 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.635745    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:13.714816 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.714792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume\") pod \"b09a7f6e-0c80-498e-add7-740c3a8ca998\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") "
Sep 09 13:34:13.714916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.714868    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxpbj\" (UniqueName: \"kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj\") pod \"b09a7f6e-0c80-498e-add7-740c3a8ca998\" (UID: \"b09a7f6e-0c80-498e-add7-740c3a8ca998\") "
Sep 09 13:34:13.714916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.714901    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtb4z\" (UniqueName: \"kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z\") pod \"14377021-bc79-4103-bdd9-662aef02cc4f\" (UID: \"14377021-bc79-4103-bdd9-662aef02cc4f\") "
Sep 09 13:34:13.715040 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.715025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.715105 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.715055    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jxcz\" (UniqueName: \"kubernetes.io/projected/22dc8db1-1151-42d3-b76a-4846747bcd9b-kube-api-access-6jxcz\") pod \"limitador-operator-controller-manager-74495c69f8-pqqzl\" (UID: \"22dc8db1-1151-42d3-b76a-4846747bcd9b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:13.715503 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.715193    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwlhg\" (UniqueName: \"kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.715503 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.715342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b09a7f6e-0c80-498e-add7-740c3a8ca998" (UID: "b09a7f6e-0c80-498e-add7-740c3a8ca998"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:34:13.715503 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.715472    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.716977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.716954    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj" (OuterVolumeSpecName: "kube-api-access-xxpbj") pod "b09a7f6e-0c80-498e-add7-740c3a8ca998" (UID: "b09a7f6e-0c80-498e-add7-740c3a8ca998"). InnerVolumeSpecName "kube-api-access-xxpbj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:34:13.717082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.717051    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z" (OuterVolumeSpecName: "kube-api-access-gtb4z") pod "14377021-bc79-4103-bdd9-662aef02cc4f" (UID: "14377021-bc79-4103-bdd9-662aef02cc4f"). InnerVolumeSpecName "kube-api-access-gtb4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:34:13.738118 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.738094    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwlhg\" (UniqueName: \"kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-8nk8p\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.738471 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.738453    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jxcz\" (UniqueName: \"kubernetes.io/projected/22dc8db1-1151-42d3-b76a-4846747bcd9b-kube-api-access-6jxcz\") pod \"limitador-operator-controller-manager-74495c69f8-pqqzl\" (UID: \"22dc8db1-1151-42d3-b76a-4846747bcd9b\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:13.807150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.807120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:13.816365 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.816345    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxpbj\" (UniqueName: \"kubernetes.io/projected/b09a7f6e-0c80-498e-add7-740c3a8ca998-kube-api-access-xxpbj\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:13.816426 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.816368    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtb4z\" (UniqueName: \"kubernetes.io/projected/14377021-bc79-4103-bdd9-662aef02cc4f-kube-api-access-gtb4z\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:13.816426 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.816378    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b09a7f6e-0c80-498e-add7-740c3a8ca998-extensions-socket-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:13.825466 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.825439    2559 generic.go:356] "Generic (PLEG): container finished" podID="b09a7f6e-0c80-498e-add7-740c3a8ca998" containerID="3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2" exitCode=0
Sep 09 13:34:13.825548 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.825486    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bwwrv"
Sep 09 13:34:13.825548 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.825523    2559 scope.go:117] "RemoveContainer" containerID="3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2"
Sep 09 13:34:13.827159 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.827136    2559 generic.go:356] "Generic (PLEG): container finished" podID="14377021-bc79-4103-bdd9-662aef02cc4f" containerID="8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d" exitCode=0
Sep 09 13:34:13.827241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.827177    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-l8ts5"
Sep 09 13:34:13.835677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.835653    2559 scope.go:117] "RemoveContainer" containerID="3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2"
Sep 09 13:34:13.836083 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.836044    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2\": container with ID starting with 3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2 not found: ID does not exist" containerID="3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2"
Sep 09 13:34:13.836151 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.836081    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2"} err="failed to get container status \"3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2\": rpc error: code = NotFound desc = could not find container \"3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2\": container with ID starting with 3e2f3f4ce817cb074a3a44e960f1f3b55cb96c86538d81bffb6bcd47638d62d2 not found: ID does not exist"
Sep 09 13:34:13.836151 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.836106    2559 scope.go:117] "RemoveContainer" containerID="8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d"
Sep 09 13:34:13.838384 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.838366    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:13.844472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.844449    2559 scope.go:117] "RemoveContainer" containerID="8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d"
Sep 09 13:34:13.844796 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:13.844777    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d\": container with ID starting with 8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d not found: ID does not exist" containerID="8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d"
Sep 09 13:34:13.844871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:13.844807    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d"} err="failed to get container status \"8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d\": rpc error: code = NotFound desc = could not find container \"8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d\": container with ID starting with 8a1d8321a0fa46133165c351455f2ff65f71a40750103c8293817a77298e089d not found: ID does not exist"
Sep 09 13:34:14.093978 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.093956    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:14.096286 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:14.096251    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86fb2613_88ae_4356_b474_4ed0ab001f96.slice/crio-a1b47058a385d148097212fe81683dc5039994e017cc8b0bcf5f10aebfab68b2 WatchSource:0}: Error finding container a1b47058a385d148097212fe81683dc5039994e017cc8b0bcf5f10aebfab68b2: Status 404 returned error can't find the container with id a1b47058a385d148097212fe81683dc5039994e017cc8b0bcf5f10aebfab68b2
Sep 09 13:34:14.126459 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.126438    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"]
Sep 09 13:34:14.130645 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:14.130620    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22dc8db1_1151_42d3_b76a_4846747bcd9b.slice/crio-846c37ada5a09fcf0fa7345d7e1ce0aaedae41477bcdafa92a9956b84842da35 WatchSource:0}: Error finding container 846c37ada5a09fcf0fa7345d7e1ce0aaedae41477bcdafa92a9956b84842da35: Status 404 returned error can't find the container with id 846c37ada5a09fcf0fa7345d7e1ce0aaedae41477bcdafa92a9956b84842da35
Sep 09 13:34:14.834841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.834807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl" event={"ID":"22dc8db1-1151-42d3-b76a-4846747bcd9b","Type":"ContainerStarted","Data":"926fb6815951eedf98efb828a58ee53edaaecfec212475b8701278a1ca7afff6"}
Sep 09 13:34:14.835247 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.834863    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl" event={"ID":"22dc8db1-1151-42d3-b76a-4846747bcd9b","Type":"ContainerStarted","Data":"846c37ada5a09fcf0fa7345d7e1ce0aaedae41477bcdafa92a9956b84842da35"}
Sep 09 13:34:14.835247 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.834971    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:14.837693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.837656    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" event={"ID":"86fb2613-88ae-4356-b474-4ed0ab001f96","Type":"ContainerStarted","Data":"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"}
Sep 09 13:34:14.837828 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.837700    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" event={"ID":"86fb2613-88ae-4356-b474-4ed0ab001f96","Type":"ContainerStarted","Data":"a1b47058a385d148097212fe81683dc5039994e017cc8b0bcf5f10aebfab68b2"}
Sep 09 13:34:14.837828 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.837752    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:14.863633 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.863586    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl" podStartSLOduration=1.863559919 podStartE2EDuration="1.863559919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:14.860212074 +0000 UTC m=+473.748912065" watchObservedRunningTime="2026-09-09 13:34:14.863559919 +0000 UTC m=+473.752259908"
Sep 09 13:34:14.896794 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:14.896744    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" podStartSLOduration=1.896732896 podStartE2EDuration="1.896732896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:14.894001676 +0000 UTC m=+473.782701666" watchObservedRunningTime="2026-09-09 13:34:14.896732896 +0000 UTC m=+473.785432886"
Sep 09 13:34:15.052432 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.052399    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:34:15.055918 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.055901    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.076140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.076113    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:34:15.127031 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.126955    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.127031 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.126998    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82sj5\" (UniqueName: \"kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.228263 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.228231    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.228401 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.228275    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82sj5\" (UniqueName: \"kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.228688 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.228665    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.248153 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.248131    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82sj5\" (UniqueName: \"kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-ltbtp\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.366973 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.366947    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:15.617132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.617093    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14377021-bc79-4103-bdd9-662aef02cc4f" path="/var/lib/kubelet/pods/14377021-bc79-4103-bdd9-662aef02cc4f/volumes"
Sep 09 13:34:15.617507 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.617490    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b09a7f6e-0c80-498e-add7-740c3a8ca998" path="/var/lib/kubelet/pods/b09a7f6e-0c80-498e-add7-740c3a8ca998/volumes"
Sep 09 13:34:15.656288 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:15.656255    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44bd4fa_16b0_4040_a1d7_4109a3ebce97.slice/crio-ecb655fe574ac11710c0d35c6a448f89882a3c24a2ec84271e1535bffcc21757 WatchSource:0}: Error finding container ecb655fe574ac11710c0d35c6a448f89882a3c24a2ec84271e1535bffcc21757: Status 404 returned error can't find the container with id ecb655fe574ac11710c0d35c6a448f89882a3c24a2ec84271e1535bffcc21757
Sep 09 13:34:15.657113 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.657086    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:34:15.851211 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.851121    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" event={"ID":"e44bd4fa-16b0-4040-a1d7-4109a3ebce97","Type":"ContainerStarted","Data":"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"}
Sep 09 13:34:15.851614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.851275    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" event={"ID":"e44bd4fa-16b0-4040-a1d7-4109a3ebce97","Type":"ContainerStarted","Data":"ecb655fe574ac11710c0d35c6a448f89882a3c24a2ec84271e1535bffcc21757"}
Sep 09 13:34:15.896381 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:15.896334    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" podStartSLOduration=0.896319565 podStartE2EDuration="896.319565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:15.895176385 +0000 UTC m=+474.783876375" watchObservedRunningTime="2026-09-09 13:34:15.896319565 +0000 UTC m=+474.785019554"
Sep 09 13:34:16.856789 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:16.856751    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:25.852937 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:25.852898    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pqqzl"
Sep 09 13:34:25.853392 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:25.853191    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:27.865077 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:27.865040    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:34:27.925410 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:27.925375    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:27.925619 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:27.925598    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" podUID="86fb2613-88ae-4356-b474-4ed0ab001f96" containerName="manager" containerID="cri-o://de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833" gracePeriod=10
Sep 09 13:34:28.168807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.168779    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:28.331955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.331925    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nwlhg\" (UniqueName: \"kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg\") pod \"86fb2613-88ae-4356-b474-4ed0ab001f96\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") "
Sep 09 13:34:28.332151 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.332034    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume\") pod \"86fb2613-88ae-4356-b474-4ed0ab001f96\" (UID: \"86fb2613-88ae-4356-b474-4ed0ab001f96\") "
Sep 09 13:34:28.332368 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.332345    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "86fb2613-88ae-4356-b474-4ed0ab001f96" (UID: "86fb2613-88ae-4356-b474-4ed0ab001f96"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:34:28.333875 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.333842    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg" (OuterVolumeSpecName: "kube-api-access-nwlhg") pod "86fb2613-88ae-4356-b474-4ed0ab001f96" (UID: "86fb2613-88ae-4356-b474-4ed0ab001f96"). InnerVolumeSpecName "kube-api-access-nwlhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:34:28.433248 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.433192    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/86fb2613-88ae-4356-b474-4ed0ab001f96-extensions-socket-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:28.433248 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.433213    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nwlhg\" (UniqueName: \"kubernetes.io/projected/86fb2613-88ae-4356-b474-4ed0ab001f96-kube-api-access-nwlhg\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:28.923834 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.923801    2559 generic.go:356] "Generic (PLEG): container finished" podID="86fb2613-88ae-4356-b474-4ed0ab001f96" containerID="de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833" exitCode=0
Sep 09 13:34:28.924230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.923842    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"
Sep 09 13:34:28.924230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.923874    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" event={"ID":"86fb2613-88ae-4356-b474-4ed0ab001f96","Type":"ContainerDied","Data":"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"}
Sep 09 13:34:28.924230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.923920    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p" event={"ID":"86fb2613-88ae-4356-b474-4ed0ab001f96","Type":"ContainerDied","Data":"a1b47058a385d148097212fe81683dc5039994e017cc8b0bcf5f10aebfab68b2"}
Sep 09 13:34:28.924230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.923945    2559 scope.go:117] "RemoveContainer" containerID="de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"
Sep 09 13:34:28.933108 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.932920    2559 scope.go:117] "RemoveContainer" containerID="de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"
Sep 09 13:34:28.933197 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:28.933179    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833\": container with ID starting with de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833 not found: ID does not exist" containerID="de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"
Sep 09 13:34:28.933242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.933206    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833"} err="failed to get container status \"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833\": rpc error: code = NotFound desc = could not find container \"de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833\": container with ID starting with de6e55b275096a5bcf1a89ae0efad84748618d6170f935709f0f944b9f56d833 not found: ID does not exist"
Sep 09 13:34:28.949220 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.949190    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:28.954061 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:28.954039    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-8nk8p"]
Sep 09 13:34:29.616479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:29.616447    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86fb2613-88ae-4356-b474-4ed0ab001f96" path="/var/lib/kubelet/pods/86fb2613-88ae-4356-b474-4ed0ab001f96/volumes"
Sep 09 13:34:44.350804 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.350767    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"]
Sep 09 13:34:44.351339 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.351322    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86fb2613-88ae-4356-b474-4ed0ab001f96" containerName="manager"
Sep 09 13:34:44.351395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.351342    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="86fb2613-88ae-4356-b474-4ed0ab001f96" containerName="manager"
Sep 09 13:34:44.351434 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.351422    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="86fb2613-88ae-4356-b474-4ed0ab001f96" containerName="manager"
Sep 09 13:34:44.373694 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.373666    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"]
Sep 09 13:34:44.373872 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.373786    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.377775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.377748    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 13:34:44.378031 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.378014    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9vbw2\""
Sep 09 13:34:44.457628 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457597    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411812a1-de71-41c8-8f62-e0e65a822c8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457634    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcg7q\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-kube-api-access-mcg7q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457657    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457676    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457768 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457773    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457802    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457855    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.457907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.457891    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559030 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.558996    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411812a1-de71-41c8-8f62-e0e65a822c8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559030 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559033    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcg7q\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-kube-api-access-mcg7q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559054    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559070    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559105    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559132    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559240 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559227    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559582 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559266    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559648 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559560    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559648 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559631    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559760 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559732    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559920 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559899    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.559965 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.559920    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/411812a1-de71-41c8-8f62-e0e65a822c8b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.561396 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.561376    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.561741 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.561719    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.589124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.589099    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcg7q\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-kube-api-access-mcg7q\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.589239 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.589154    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/411812a1-de71-41c8-8f62-e0e65a822c8b-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-dldkt\" (UID: \"411812a1-de71-41c8-8f62-e0e65a822c8b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.685263 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.685194    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:44.974744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:44.974715    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"]
Sep 09 13:34:44.975839 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:44.975810    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411812a1_de71_41c8_8f62_e0e65a822c8b.slice/crio-052b90e25a762858c862d12ff01c2005c62484a286cfecc1140443724f035072 WatchSource:0}: Error finding container 052b90e25a762858c862d12ff01c2005c62484a286cfecc1140443724f035072: Status 404 returned error can't find the container with id 052b90e25a762858c862d12ff01c2005c62484a286cfecc1140443724f035072
Sep 09 13:34:45.010924 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:45.010898    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" event={"ID":"411812a1-de71-41c8-8f62-e0e65a822c8b","Type":"ContainerStarted","Data":"052b90e25a762858c862d12ff01c2005c62484a286cfecc1140443724f035072"}
Sep 09 13:34:48.053561 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:48.053521    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:34:48.053935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:48.053629    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:34:48.053935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:48.053700    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:34:49.033860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:49.033818    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" event={"ID":"411812a1-de71-41c8-8f62-e0e65a822c8b","Type":"ContainerStarted","Data":"19201e611304212e38fbb09324bcf8bddd77f7134310eda5b71c21ebfb5d2870"}
Sep 09 13:34:49.054525 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:49.054475    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podStartSLOduration=1.978974267 podStartE2EDuration="5.054455697s" totalImagesPullingTime="3.07548143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:49.054028306 +0000 UTC m=+507.942728299" watchObservedRunningTime="2026-09-09 13:34:49.054455697 +0000 UTC m=+507.943155688"
Sep 09 13:34:49.686271 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:49.686236    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:49.690791 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:49.690766    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:50.040004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.039976    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:50.041025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.041005    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:34:50.111986 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.111930    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:34:50.115750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.115731    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.118143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.118124    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 13:34:50.126905 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.126886    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:34:50.209129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.209097    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:34:50.210760 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.210727    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.210910 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.210818    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzv4d\" (UniqueName: \"kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.312126 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.312034    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.312317 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.312140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzv4d\" (UniqueName: \"kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.312954 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.312929    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.320156 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.320133    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzv4d\" (UniqueName: \"kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d\") pod \"limitador-limitador-8588749977-2klqt\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") " pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.427914 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.427875    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:34:50.707309 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.707284    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:34:50.708625 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:50.708567    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b56d922_ccf9_43de_b7c5_fd848f3f7348.slice/crio-472ce79bfe89c04f34af1a9a37a81872d3caf22b44a95bbc0b0511b81b56aed7 WatchSource:0}: Error finding container 472ce79bfe89c04f34af1a9a37a81872d3caf22b44a95bbc0b0511b81b56aed7: Status 404 returned error can't find the container with id 472ce79bfe89c04f34af1a9a37a81872d3caf22b44a95bbc0b0511b81b56aed7
Sep 09 13:34:50.903152 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.903118    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:50.908129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.908096    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:50.910819 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.910797    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pdffk\""
Sep 09 13:34:50.916946 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:50.916918    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:51.019140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.019104    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqtdp\" (UniqueName: \"kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp\") pod \"authorino-7bcfd8f9cd-6fnrf\" (UID: \"753737d8-5b7b-42e7-baaa-8e095e5d1c9d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:51.046923 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.046884    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" event={"ID":"1b56d922-ccf9-43de-b7c5-fd848f3f7348","Type":"ContainerStarted","Data":"472ce79bfe89c04f34af1a9a37a81872d3caf22b44a95bbc0b0511b81b56aed7"}
Sep 09 13:34:51.120179 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.120142    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqtdp\" (UniqueName: \"kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp\") pod \"authorino-7bcfd8f9cd-6fnrf\" (UID: \"753737d8-5b7b-42e7-baaa-8e095e5d1c9d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:51.132025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.132001    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqtdp\" (UniqueName: \"kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp\") pod \"authorino-7bcfd8f9cd-6fnrf\" (UID: \"753737d8-5b7b-42e7-baaa-8e095e5d1c9d\") " pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:51.218483 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.218448    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:51.495421 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:51.495393    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:51.496693 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:34:51.496668    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod753737d8_5b7b_42e7_baaa_8e095e5d1c9d.slice/crio-9ce91a37bc40a28ee6f0c3351667d93a6b361c3d6b96dfd462cb23bdd16c60a7 WatchSource:0}: Error finding container 9ce91a37bc40a28ee6f0c3351667d93a6b361c3d6b96dfd462cb23bdd16c60a7: Status 404 returned error can't find the container with id 9ce91a37bc40a28ee6f0c3351667d93a6b361c3d6b96dfd462cb23bdd16c60a7
Sep 09 13:34:52.054021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:52.053984    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" event={"ID":"753737d8-5b7b-42e7-baaa-8e095e5d1c9d","Type":"ContainerStarted","Data":"9ce91a37bc40a28ee6f0c3351667d93a6b361c3d6b96dfd462cb23bdd16c60a7"}
Sep 09 13:34:54.068008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:54.067971    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" event={"ID":"753737d8-5b7b-42e7-baaa-8e095e5d1c9d","Type":"ContainerStarted","Data":"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"}
Sep 09 13:34:54.542310 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:54.542253    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" podStartSLOduration=2.85768635 podStartE2EDuration="4.542236588s" totalImagesPullingTime="1.684550238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:34:54.086344747 +0000 UTC m=+512.975044736" watchObservedRunningTime="2026-09-09 13:34:54.542236588 +0000 UTC m=+513.430936578"
Sep 09 13:34:54.542963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:54.542945    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:56.079126 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:56.079087    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" podUID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" containerName="authorino" containerID="cri-o://3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f" gracePeriod=30
Sep 09 13:34:56.319835 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:56.319812    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:56.364826 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:56.364760    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqtdp\" (UniqueName: \"kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp\") pod \"753737d8-5b7b-42e7-baaa-8e095e5d1c9d\" (UID: \"753737d8-5b7b-42e7-baaa-8e095e5d1c9d\") "
Sep 09 13:34:56.366747 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:56.366722    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp" (OuterVolumeSpecName: "kube-api-access-qqtdp") pod "753737d8-5b7b-42e7-baaa-8e095e5d1c9d" (UID: "753737d8-5b7b-42e7-baaa-8e095e5d1c9d"). InnerVolumeSpecName "kube-api-access-qqtdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:34:56.466044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:56.466021    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqtdp\" (UniqueName: \"kubernetes.io/projected/753737d8-5b7b-42e7-baaa-8e095e5d1c9d-kube-api-access-qqtdp\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:34:57.085748 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.085717    2559 generic.go:356] "Generic (PLEG): container finished" podID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" containerID="3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f" exitCode=0
Sep 09 13:34:57.086219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.085753    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf"
Sep 09 13:34:57.086219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.085774    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" event={"ID":"753737d8-5b7b-42e7-baaa-8e095e5d1c9d","Type":"ContainerDied","Data":"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"}
Sep 09 13:34:57.086219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.085807    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-6fnrf" event={"ID":"753737d8-5b7b-42e7-baaa-8e095e5d1c9d","Type":"ContainerDied","Data":"9ce91a37bc40a28ee6f0c3351667d93a6b361c3d6b96dfd462cb23bdd16c60a7"}
Sep 09 13:34:57.086219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.085823    2559 scope.go:117] "RemoveContainer" containerID="3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"
Sep 09 13:34:57.094975 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.094956    2559 scope.go:117] "RemoveContainer" containerID="3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"
Sep 09 13:34:57.095244 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:34:57.095220    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f\": container with ID starting with 3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f not found: ID does not exist" containerID="3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"
Sep 09 13:34:57.095314 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.095250    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f"} err="failed to get container status \"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f\": rpc error: code = NotFound desc = could not find container \"3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f\": container with ID starting with 3ec5746b46a7b977e55508cf4805e1dc4b07ea118e44ddc9935f25b9f070a20f not found: ID does not exist"
Sep 09 13:34:57.109148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.109124    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:57.111101 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.111075    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-6fnrf"]
Sep 09 13:34:57.616658 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:34:57.616628    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" path="/var/lib/kubelet/pods/753737d8-5b7b-42e7-baaa-8e095e5d1c9d/volumes"
Sep 09 13:35:01.111199 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:01.111115    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" event={"ID":"1b56d922-ccf9-43de-b7c5-fd848f3f7348","Type":"ContainerStarted","Data":"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"}
Sep 09 13:35:01.111524 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:01.111233    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:35:01.132607 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:01.132548    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" podStartSLOduration=1.112995228 podStartE2EDuration="11.132534673s" totalImagesPullingTime="10.019539445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:35:01.130672189 +0000 UTC m=+520.019372173" watchObservedRunningTime="2026-09-09 13:35:01.132534673 +0000 UTC m=+520.021234662"
Sep 09 13:35:11.833669 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:11.833638    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:35:11.834152 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:11.833850    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" podUID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" containerName="limitador" containerID="cri-o://edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e" gracePeriod=30
Sep 09 13:35:11.835014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:11.834987    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:35:12.377236 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.377212    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:35:12.495294 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.495269    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzv4d\" (UniqueName: \"kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d\") pod \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") "
Sep 09 13:35:12.495427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.495352    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file\") pod \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\" (UID: \"1b56d922-ccf9-43de-b7c5-fd848f3f7348\") "
Sep 09 13:35:12.495735 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.495712    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file" (OuterVolumeSpecName: "config-file") pod "1b56d922-ccf9-43de-b7c5-fd848f3f7348" (UID: "1b56d922-ccf9-43de-b7c5-fd848f3f7348"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:35:12.497259 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.497239    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d" (OuterVolumeSpecName: "kube-api-access-fzv4d") pod "1b56d922-ccf9-43de-b7c5-fd848f3f7348" (UID: "1b56d922-ccf9-43de-b7c5-fd848f3f7348"). InnerVolumeSpecName "kube-api-access-fzv4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:35:12.596485 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.596457    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzv4d\" (UniqueName: \"kubernetes.io/projected/1b56d922-ccf9-43de-b7c5-fd848f3f7348-kube-api-access-fzv4d\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:12.596485 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:12.596482    2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/1b56d922-ccf9-43de-b7c5-fd848f3f7348-config-file\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:13.177863 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.177826    2559 generic.go:356] "Generic (PLEG): container finished" podID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" containerID="edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e" exitCode=0
Sep 09 13:35:13.178505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.177886    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-2klqt"
Sep 09 13:35:13.178505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.177909    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" event={"ID":"1b56d922-ccf9-43de-b7c5-fd848f3f7348","Type":"ContainerDied","Data":"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"}
Sep 09 13:35:13.178505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.177982    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-2klqt" event={"ID":"1b56d922-ccf9-43de-b7c5-fd848f3f7348","Type":"ContainerDied","Data":"472ce79bfe89c04f34af1a9a37a81872d3caf22b44a95bbc0b0511b81b56aed7"}
Sep 09 13:35:13.178505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.178005    2559 scope.go:117] "RemoveContainer" containerID="edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"
Sep 09 13:35:13.187097 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.187074    2559 scope.go:117] "RemoveContainer" containerID="edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"
Sep 09 13:35:13.187378 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:35:13.187356    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e\": container with ID starting with edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e not found: ID does not exist" containerID="edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"
Sep 09 13:35:13.187422 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.187387    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e"} err="failed to get container status \"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e\": rpc error: code = NotFound desc = could not find container \"edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e\": container with ID starting with edc282b7b180e49a734d89a1c81f205d500d0537db6f3e2d0edc365885a7b37e not found: ID does not exist"
Sep 09 13:35:13.201136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.201111    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:35:13.207952 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.207932    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-2klqt"]
Sep 09 13:35:13.617480 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:13.617447    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" path="/var/lib/kubelet/pods/1b56d922-ccf9-43de-b7c5-fd848f3f7348/volumes"
Sep 09 13:35:17.061181 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061110    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"]
Sep 09 13:35:17.061538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061469    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" containerName="authorino"
Sep 09 13:35:17.061538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061479    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" containerName="authorino"
Sep 09 13:35:17.061538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061491    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" containerName="limitador"
Sep 09 13:35:17.061538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061496    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" containerName="limitador"
Sep 09 13:35:17.061724 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061585    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b56d922-ccf9-43de-b7c5-fd848f3f7348" containerName="limitador"
Sep 09 13:35:17.061724 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.061594    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="753737d8-5b7b-42e7-baaa-8e095e5d1c9d" containerName="authorino"
Sep 09 13:35:17.063810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.063793    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.066436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.066406    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 13:35:17.066436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.066434    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-wqtk8\""
Sep 09 13:35:17.067458 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.067444    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 13:35:17.067519 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.067444    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 13:35:17.078858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.078838    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"]
Sep 09 13:35:17.234971 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.234940    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-data\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.235132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.235006    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqnkz\" (UniqueName: \"kubernetes.io/projected/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-kube-api-access-hqnkz\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.336239 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.336162    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-data\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.336381 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.336247    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqnkz\" (UniqueName: \"kubernetes.io/projected/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-kube-api-access-hqnkz\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.336615 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.336568    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-data\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.344595 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.344559    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqnkz\" (UniqueName: \"kubernetes.io/projected/2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12-kube-api-access-hqnkz\") pod \"postgres-69fb4594cf-crtxx\" (UID: \"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.373497 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.373476    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:17.645928 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:17.645904    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"]
Sep 09 13:35:17.647169 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:35:17.647142    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db5bf17_2f8b_4d3d_a2b2_5c4131db1b12.slice/crio-a5c6ea1f945a3b6b5d26e1003e4ba01b34c98525957bbf3f616dcfa430dc3ff1 WatchSource:0}: Error finding container a5c6ea1f945a3b6b5d26e1003e4ba01b34c98525957bbf3f616dcfa430dc3ff1: Status 404 returned error can't find the container with id a5c6ea1f945a3b6b5d26e1003e4ba01b34c98525957bbf3f616dcfa430dc3ff1
Sep 09 13:35:18.207521 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:18.207488    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx" event={"ID":"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12","Type":"ContainerStarted","Data":"a5c6ea1f945a3b6b5d26e1003e4ba01b34c98525957bbf3f616dcfa430dc3ff1"}
Sep 09 13:35:21.751916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:21.751891    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 13:35:22.232143 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:22.232111    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx" event={"ID":"2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12","Type":"ContainerStarted","Data":"a306b4403dc4fc78ae0dd41496b1ed2be39b316eda59e1e1e32ddde92af5d603"}
Sep 09 13:35:22.232331 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:22.232196    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:22.251813 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:22.251770    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx" podStartSLOduration=1.15082789 podStartE2EDuration="5.251757559s" totalImagesPullingTime="4.100929669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:35:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:35:22.248712139 +0000 UTC m=+541.137412129" watchObservedRunningTime="2026-09-09 13:35:22.251757559 +0000 UTC m=+541.140457578"
Sep 09 13:35:28.266933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:28.266902    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-crtxx"
Sep 09 13:35:28.893279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:28.893249    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-gsgf8"]
Sep 09 13:35:28.897960 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:28.897938    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:28.901440 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:28.901405    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pdffk\""
Sep 09 13:35:28.908834 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:28.908810    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-gsgf8"]
Sep 09 13:35:29.024049 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.024014    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gsgf8"]
Sep 09 13:35:29.024282 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:35:29.024262    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xkd4h], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-gsgf8" podUID="ae3d266f-ee2a-4934-b888-b836ce94879a"
Sep 09 13:35:29.035920 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.035888    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkd4h\" (UniqueName: \"kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h\") pod \"authorino-557f876f69-gsgf8\" (UID: \"ae3d266f-ee2a-4934-b888-b836ce94879a\") " pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:29.062505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.062482    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-mf8rv"]
Sep 09 13:35:29.066060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.066044    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.069284 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.069261    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 13:35:29.081168 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.081143    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-mf8rv"]
Sep 09 13:35:29.136626 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.136592    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkd4h\" (UniqueName: \"kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h\") pod \"authorino-557f876f69-gsgf8\" (UID: \"ae3d266f-ee2a-4934-b888-b836ce94879a\") " pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:29.140316 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.140296    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-mf8rv"]
Sep 09 13:35:29.140518 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:35:29.140501    2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-t6djx tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-t6djx tls-cert]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-mf8rv" podUID="90782cc6-cdb9-4bce-b0bc-b1c47b981d2b"
Sep 09 13:35:29.148039 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.147985    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkd4h\" (UniqueName: \"kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h\") pod \"authorino-557f876f69-gsgf8\" (UID: \"ae3d266f-ee2a-4934-b888-b836ce94879a\") " pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:29.180295 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.180265    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:35:29.183756 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.183736    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.203193 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.203172    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:35:29.237388 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.237360    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6djx\" (UniqueName: \"kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.237492 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.237415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.271853 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.271830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.272145 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.271836    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:29.278698 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.278682    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.281792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.281776    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:29.338082 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.338059    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6djx\" (UniqueName: \"kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.338215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.338140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.338215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.338182    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql8z9\" (UniqueName: \"kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.338334 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.338223    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.340660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.340640    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.347526 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.347507    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6djx\" (UniqueName: \"kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx\") pod \"authorino-6d4dfc649f-mf8rv\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") " pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:29.438718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.438649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert\") pod \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") "
Sep 09 13:35:29.438718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.438713    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkd4h\" (UniqueName: \"kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h\") pod \"ae3d266f-ee2a-4934-b888-b836ce94879a\" (UID: \"ae3d266f-ee2a-4934-b888-b836ce94879a\") "
Sep 09 13:35:29.438869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.438838    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql8z9\" (UniqueName: \"kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.438922 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.438871    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.440767 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.440731    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h" (OuterVolumeSpecName: "kube-api-access-xkd4h") pod "ae3d266f-ee2a-4934-b888-b836ce94879a" (UID: "ae3d266f-ee2a-4934-b888-b836ce94879a"). InnerVolumeSpecName "kube-api-access-xkd4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:35:29.440869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.440763    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "90782cc6-cdb9-4bce-b0bc-b1c47b981d2b" (UID: "90782cc6-cdb9-4bce-b0bc-b1c47b981d2b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:35:29.441129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.441113    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.448640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.448622    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql8z9\" (UniqueName: \"kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9\") pod \"authorino-5b5dd7b4f5-7cndd\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") " pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.492481 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.492462    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:35:29.539882 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.539847    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6djx\" (UniqueName: \"kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx\") pod \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\" (UID: \"90782cc6-cdb9-4bce-b0bc-b1c47b981d2b\") "
Sep 09 13:35:29.540230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.540210    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-tls-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:29.540325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.540235    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkd4h\" (UniqueName: \"kubernetes.io/projected/ae3d266f-ee2a-4934-b888-b836ce94879a-kube-api-access-xkd4h\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:29.542241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.542213    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx" (OuterVolumeSpecName: "kube-api-access-t6djx") pod "90782cc6-cdb9-4bce-b0bc-b1c47b981d2b" (UID: "90782cc6-cdb9-4bce-b0bc-b1c47b981d2b"). InnerVolumeSpecName "kube-api-access-t6djx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:35:29.640661 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.640635    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6djx\" (UniqueName: \"kubernetes.io/projected/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b-kube-api-access-t6djx\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:29.773990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:29.773968    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:35:29.774776 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:35:29.774757    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0c4a8b5_232c_48c3_b7aa_7b9ce7d1eccb.slice/crio-34a26ba7de1dcd80415ed2638e2ae003486e3e2c91eda9dde596628e256d2405 WatchSource:0}: Error finding container 34a26ba7de1dcd80415ed2638e2ae003486e3e2c91eda9dde596628e256d2405: Status 404 returned error can't find the container with id 34a26ba7de1dcd80415ed2638e2ae003486e3e2c91eda9dde596628e256d2405
Sep 09 13:35:30.278552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.278514    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" event={"ID":"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb","Type":"ContainerStarted","Data":"34a26ba7de1dcd80415ed2638e2ae003486e3e2c91eda9dde596628e256d2405"}
Sep 09 13:35:30.278552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.278541    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-mf8rv"
Sep 09 13:35:30.278933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.278541    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-gsgf8"
Sep 09 13:35:30.313757 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.313727    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-gsgf8"]
Sep 09 13:35:30.320481 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.320457    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-gsgf8"]
Sep 09 13:35:30.344060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.344034    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-mf8rv"]
Sep 09 13:35:30.353696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:30.353673    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-mf8rv"]
Sep 09 13:35:31.617925 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:31.617889    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90782cc6-cdb9-4bce-b0bc-b1c47b981d2b" path="/var/lib/kubelet/pods/90782cc6-cdb9-4bce-b0bc-b1c47b981d2b/volumes"
Sep 09 13:35:31.618312 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:31.618105    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae3d266f-ee2a-4934-b888-b836ce94879a" path="/var/lib/kubelet/pods/ae3d266f-ee2a-4934-b888-b836ce94879a/volumes"
Sep 09 13:35:33.302091 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:33.302060    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" event={"ID":"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb","Type":"ContainerStarted","Data":"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"}
Sep 09 13:35:33.319801 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:33.319752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" podStartSLOduration=1.727399624 podStartE2EDuration="4.319738254s" totalImagesPullingTime="2.59233863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:35:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:35:33.317039585 +0000 UTC m=+552.205739575" watchObservedRunningTime="2026-09-09 13:35:33.319738254 +0000 UTC m=+552.208438244"
Sep 09 13:35:38.427020 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.426982    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:38.430282 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.430253    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.432956 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.432934    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 13:35:38.433065 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.433008    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8lp8w\""
Sep 09 13:35:38.433499 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.433480    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 13:35:38.434120 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.434099    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:35:38.442089 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.442066    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:38.511728 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.511699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgngv\" (UniqueName: \"kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.511882 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.511738    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.511882 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.511770    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.613176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.613140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.613329 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.613195    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.613329 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.613276    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgngv\" (UniqueName: \"kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.615893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.615864    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.616015 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.615908    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.623489 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.623464    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgngv\" (UniqueName: \"kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv\") pod \"maas-controller-55cb885958-s4xlm\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") " pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.628859 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.628836    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:35:38.631532 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.631515    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.642521 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.642495    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:35:38.714547 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.714523    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4f6tt\" (UniqueName: \"kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.714654 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.714556    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.714654 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.714588    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.742974 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.742952    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:38.815358 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.815324    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4f6tt\" (UniqueName: \"kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.815491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.815373    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.815491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.815404    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.818465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.818435    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.819056 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.819024    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.824333 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.824297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4f6tt\" (UniqueName: \"kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt\") pod \"maas-controller-59b6f545bd-wmlcx\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") " pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:38.942423 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:38.942386    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:39.016899 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:39.016870    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:39.018083 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:35:39.018054    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe4dd1af_6898_4958_9661_4804bfb00b8b.slice/crio-f3c2be3ec5f78f7ce7e5a1a4bc072e06b04d0bf0afc890b87e81769181aaa2d0 WatchSource:0}: Error finding container f3c2be3ec5f78f7ce7e5a1a4bc072e06b04d0bf0afc890b87e81769181aaa2d0: Status 404 returned error can't find the container with id f3c2be3ec5f78f7ce7e5a1a4bc072e06b04d0bf0afc890b87e81769181aaa2d0
Sep 09 13:35:39.211287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:39.211252    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:35:39.212561 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:35:39.212541    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod171df6d2_84af_40ed_87de_befa44e517f5.slice/crio-152d90b133f62b9624143833bd7a23ac26ed70fca1d5386a1adb51c30efe1cb1 WatchSource:0}: Error finding container 152d90b133f62b9624143833bd7a23ac26ed70fca1d5386a1adb51c30efe1cb1: Status 404 returned error can't find the container with id 152d90b133f62b9624143833bd7a23ac26ed70fca1d5386a1adb51c30efe1cb1
Sep 09 13:35:39.338498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:39.338420    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" event={"ID":"171df6d2-84af-40ed-87de-befa44e517f5","Type":"ContainerStarted","Data":"152d90b133f62b9624143833bd7a23ac26ed70fca1d5386a1adb51c30efe1cb1"}
Sep 09 13:35:39.339685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:39.339657    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cb885958-s4xlm" event={"ID":"be4dd1af-6898-4958-9661-4804bfb00b8b","Type":"ContainerStarted","Data":"f3c2be3ec5f78f7ce7e5a1a4bc072e06b04d0bf0afc890b87e81769181aaa2d0"}
Sep 09 13:35:42.361252 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.361144    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" event={"ID":"171df6d2-84af-40ed-87de-befa44e517f5","Type":"ContainerStarted","Data":"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"}
Sep 09 13:35:42.361709 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.361286    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:42.363178 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.363147    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cb885958-s4xlm" event={"ID":"be4dd1af-6898-4958-9661-4804bfb00b8b","Type":"ContainerStarted","Data":"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"}
Sep 09 13:35:42.363306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.363197    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:42.380110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.380060    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" podStartSLOduration=1.5138232 podStartE2EDuration="4.380042499s" totalImagesPullingTime="2.866219299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:35:42.379591106 +0000 UTC m=+561.268291098" watchObservedRunningTime="2026-09-09 13:35:42.380042499 +0000 UTC m=+561.268742490"
Sep 09 13:35:42.399226 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:42.399171    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55cb885958-s4xlm" podStartSLOduration=1.789562667 podStartE2EDuration="4.399154211s" totalImagesPullingTime="2.609591544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:35:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:35:42.396815423 +0000 UTC m=+561.285515413" watchObservedRunningTime="2026-09-09 13:35:42.399154211 +0000 UTC m=+561.287854200"
Sep 09 13:35:53.371636 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.371606    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:35:53.371999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.371675    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:53.444549 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.444515    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:53.445243 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.444849    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55cb885958-s4xlm" podUID="be4dd1af-6898-4958-9661-4804bfb00b8b" containerName="manager" containerID="cri-o://7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4" gracePeriod=30
Sep 09 13:35:53.682655 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.682625    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:53.744281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.744252    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jgngv\" (UniqueName: \"kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv\") pod \"be4dd1af-6898-4958-9661-4804bfb00b8b\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") "
Sep 09 13:35:53.744427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.744289    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs\") pod \"be4dd1af-6898-4958-9661-4804bfb00b8b\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") "
Sep 09 13:35:53.744427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.744349    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert\") pod \"be4dd1af-6898-4958-9661-4804bfb00b8b\" (UID: \"be4dd1af-6898-4958-9661-4804bfb00b8b\") "
Sep 09 13:35:53.746750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.746723    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "be4dd1af-6898-4958-9661-4804bfb00b8b" (UID: "be4dd1af-6898-4958-9661-4804bfb00b8b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:35:53.746750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.746741    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "be4dd1af-6898-4958-9661-4804bfb00b8b" (UID: "be4dd1af-6898-4958-9661-4804bfb00b8b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:35:53.746941 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.746925    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv" (OuterVolumeSpecName: "kube-api-access-jgngv") pod "be4dd1af-6898-4958-9661-4804bfb00b8b" (UID: "be4dd1af-6898-4958-9661-4804bfb00b8b"). InnerVolumeSpecName "kube-api-access-jgngv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:35:53.846216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.846191    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jgngv\" (UniqueName: \"kubernetes.io/projected/be4dd1af-6898-4958-9661-4804bfb00b8b-kube-api-access-jgngv\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:53.846216 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.846213    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:53.846362 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:53.846223    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be4dd1af-6898-4958-9661-4804bfb00b8b-webhook-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:35:54.432244 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.432206    2559 generic.go:356] "Generic (PLEG): container finished" podID="be4dd1af-6898-4958-9661-4804bfb00b8b" containerID="7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4" exitCode=0
Sep 09 13:35:54.432677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.432252    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55cb885958-s4xlm"
Sep 09 13:35:54.432677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.432287    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cb885958-s4xlm" event={"ID":"be4dd1af-6898-4958-9661-4804bfb00b8b","Type":"ContainerDied","Data":"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"}
Sep 09 13:35:54.432677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.432340    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55cb885958-s4xlm" event={"ID":"be4dd1af-6898-4958-9661-4804bfb00b8b","Type":"ContainerDied","Data":"f3c2be3ec5f78f7ce7e5a1a4bc072e06b04d0bf0afc890b87e81769181aaa2d0"}
Sep 09 13:35:54.432677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.432366    2559 scope.go:117] "RemoveContainer" containerID="7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"
Sep 09 13:35:54.442124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.442109    2559 scope.go:117] "RemoveContainer" containerID="7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"
Sep 09 13:35:54.442390 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:35:54.442370    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4\": container with ID starting with 7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4 not found: ID does not exist" containerID="7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"
Sep 09 13:35:54.442446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.442397    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4"} err="failed to get container status \"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4\": rpc error: code = NotFound desc = could not find container \"7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4\": container with ID starting with 7093600b57f8262da4715bfdea2a3461ab71c19771d1227c7f4d1b73ceebbdb4 not found: ID does not exist"
Sep 09 13:35:54.453964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.453937    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:54.457644 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:54.457621    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55cb885958-s4xlm"]
Sep 09 13:35:55.616550 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:35:55.616515    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be4dd1af-6898-4958-9661-4804bfb00b8b" path="/var/lib/kubelet/pods/be4dd1af-6898-4958-9661-4804bfb00b8b/volumes"
Sep 09 13:36:14.519674 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.519641    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 13:36:14.520128 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.520063    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be4dd1af-6898-4958-9661-4804bfb00b8b" containerName="manager"
Sep 09 13:36:14.520128 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.520076    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="be4dd1af-6898-4958-9661-4804bfb00b8b" containerName="manager"
Sep 09 13:36:14.520204 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.520175    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="be4dd1af-6898-4958-9661-4804bfb00b8b" containerName="manager"
Sep 09 13:36:14.524647 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.524628    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.529264 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.529240    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 13:36:14.529538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.529522    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 13:36:14.530394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.530376    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-s8jj8\""
Sep 09 13:36:14.540081 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.540052    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 13:36:14.609849 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.609819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.609991 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.609862    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l886q\" (UniqueName: \"kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.609991 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.609960    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.610093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.609991    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.710736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.710709    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.710736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.710742    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.710964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.710775    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.710964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.710797    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l886q\" (UniqueName: \"kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.710964 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:36:14.710888    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 09 13:36:14.711148 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:36:14.710968    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls podName:940f8b83-9a42-4d17-9c06-39ac71fb9547 nodeName:}" failed. No retries permitted until 2026-09-09 13:36:15.210948261 +0000 UTC m=+594.099648237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls") pod "maas-api-6b4b89c5b5-dhs6x" (UID: "940f8b83-9a42-4d17-9c06-39ac71fb9547") : secret "maas-api-serving-cert" not found
Sep 09 13:36:14.711455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.711430    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.713148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.713129    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:14.719428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:14.719398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l886q\" (UniqueName: \"kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:15.080888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.080857    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"]
Sep 09 13:36:15.083310 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.083293    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.086402 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.086372    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 13:36:15.086402 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.086372    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-8fz64\""
Sep 09 13:36:15.095151 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.095130    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"]
Sep 09 13:36:15.102287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.102266    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"]
Sep 09 13:36:15.105600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.105552    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.119518 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.119497    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"]
Sep 09 13:36:15.215129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.215076    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a9d046-9290-433e-b8ca-a75ab25e4abd-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.215129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.215122    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmlf6\" (UniqueName: \"kubernetes.io/projected/d7375e01-5993-4511-b8bd-62c73387e3b1-kube-api-access-tmlf6\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.215395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.215146    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49bhr\" (UniqueName: \"kubernetes.io/projected/38a9d046-9290-433e-b8ca-a75ab25e4abd-kube-api-access-49bhr\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.215395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.215253    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7375e01-5993-4511-b8bd-62c73387e3b1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.215395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.215304    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:15.217820 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.217793    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") pod \"maas-api-6b4b89c5b5-dhs6x\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") " pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:15.316802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.316767    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7375e01-5993-4511-b8bd-62c73387e3b1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.317153 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.317128    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a9d046-9290-433e-b8ca-a75ab25e4abd-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.318032 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.318011    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/38a9d046-9290-433e-b8ca-a75ab25e4abd-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.318176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.317828    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d7375e01-5993-4511-b8bd-62c73387e3b1-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.318287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.318043    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmlf6\" (UniqueName: \"kubernetes.io/projected/d7375e01-5993-4511-b8bd-62c73387e3b1-kube-api-access-tmlf6\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.318431 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.318311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49bhr\" (UniqueName: \"kubernetes.io/projected/38a9d046-9290-433e-b8ca-a75ab25e4abd-kube-api-access-49bhr\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.326359 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.326334    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49bhr\" (UniqueName: \"kubernetes.io/projected/38a9d046-9290-433e-b8ca-a75ab25e4abd-kube-api-access-49bhr\") pod \"payload-processing-5b54b7fbb7-4s7cw\" (UID: \"38a9d046-9290-433e-b8ca-a75ab25e4abd\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.326479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.326395    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmlf6\" (UniqueName: \"kubernetes.io/projected/d7375e01-5993-4511-b8bd-62c73387e3b1-kube-api-access-tmlf6\") pod \"payload-pre-processing-59fbc47bf8-mz5t5\" (UID: \"d7375e01-5993-4511-b8bd-62c73387e3b1\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.394986 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.394899    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:15.415438 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.415405    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:15.437950 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.437859    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:15.688722 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.688690    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"]
Sep 09 13:36:15.689298 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:36:15.689275    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7375e01_5993_4511_b8bd_62c73387e3b1.slice/crio-3eff41df12662e03370230996e01eb4ba7152c8cd6a0dff9f5260881a29ffad2 WatchSource:0}: Error finding container 3eff41df12662e03370230996e01eb4ba7152c8cd6a0dff9f5260881a29ffad2: Status 404 returned error can't find the container with id 3eff41df12662e03370230996e01eb4ba7152c8cd6a0dff9f5260881a29ffad2
Sep 09 13:36:15.718882 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.718155    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"]
Sep 09 13:36:15.762411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:15.762304    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 13:36:16.565008 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:16.564960    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" event={"ID":"940f8b83-9a42-4d17-9c06-39ac71fb9547","Type":"ContainerStarted","Data":"19bfc6c7be997eec547cc1a969980b92aa813c5f088f6e24c517170598d756a5"}
Sep 09 13:36:16.568099 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:16.568069    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5" event={"ID":"d7375e01-5993-4511-b8bd-62c73387e3b1","Type":"ContainerStarted","Data":"3eff41df12662e03370230996e01eb4ba7152c8cd6a0dff9f5260881a29ffad2"}
Sep 09 13:36:16.569999 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:16.569966    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw" event={"ID":"38a9d046-9290-433e-b8ca-a75ab25e4abd","Type":"ContainerStarted","Data":"eb35d507ce517889e929f64a644fcc434b3f1e569a441b2ba2f5e584400edc83"}
Sep 09 13:36:19.594076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.594037    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5" event={"ID":"d7375e01-5993-4511-b8bd-62c73387e3b1","Type":"ContainerStarted","Data":"494ef9564ecd9edbad027a8ef736430d81dda87bba6c5f0ead2dcd523b5ae237"}
Sep 09 13:36:19.594521 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.594098    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:19.595612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.595585    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw" event={"ID":"38a9d046-9290-433e-b8ca-a75ab25e4abd","Type":"ContainerStarted","Data":"c851523eb7ffa7619b2aff546a1b4875e46cc9f647c77b31632095ebf6830415"}
Sep 09 13:36:19.595745 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.595650    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:19.597030 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.597011    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" event={"ID":"940f8b83-9a42-4d17-9c06-39ac71fb9547","Type":"ContainerStarted","Data":"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"}
Sep 09 13:36:19.597133 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.597121    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:19.635052 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.634992    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw" podStartSLOduration=1.72699231 podStartE2EDuration="4.634970579s" totalImagesPullingTime="2.907978269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:36:19.634213687 +0000 UTC m=+598.522913677" watchObservedRunningTime="2026-09-09 13:36:19.634970579 +0000 UTC m=+598.523670569"
Sep 09 13:36:19.636296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.636251    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5" podStartSLOduration=1.6889755979999999 podStartE2EDuration="4.636241251s" totalImagesPullingTime="2.947265653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:36:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:36:19.614877369 +0000 UTC m=+598.503577364" watchObservedRunningTime="2026-09-09 13:36:19.636241251 +0000 UTC m=+598.524941240"
Sep 09 13:36:19.652631 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:19.652583    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" podStartSLOduration=2.785550385 podStartE2EDuration="5.652557308s" totalImagesPullingTime="2.867006923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:36:19.650132624 +0000 UTC m=+598.538832626" watchObservedRunningTime="2026-09-09 13:36:19.652557308 +0000 UTC m=+598.541257297"
Sep 09 13:36:24.628914 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.628815    2559 generic.go:356] "Generic (PLEG): container finished" podID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerID="19201e611304212e38fbb09324bcf8bddd77f7134310eda5b71c21ebfb5d2870" exitCode=0
Sep 09 13:36:24.628914 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.628895    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" event={"ID":"411812a1-de71-41c8-8f62-e0e65a822c8b","Type":"ContainerDied","Data":"19201e611304212e38fbb09324bcf8bddd77f7134310eda5b71c21ebfb5d2870"}
Sep 09 13:36:24.629455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.629339    2559 scope.go:117] "RemoveContainer" containerID="19201e611304212e38fbb09324bcf8bddd77f7134310eda5b71c21ebfb5d2870"
Sep 09 13:36:24.630350 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.630322    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:36:24.630437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.630386    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:36:24.630437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:24.630416    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:36:25.621372 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:25.621344    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 13:36:25.637343 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:25.637309    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" event={"ID":"411812a1-de71-41c8-8f62-e0e65a822c8b","Type":"ContainerStarted","Data":"40973d5b24729acbbe773957077b49c3f452356abfdd8cc1cde1c2876d2a0552"}
Sep 09 13:36:25.686408 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:25.686377    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:36:25.686682 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:25.686656    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dldkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 13:36:25.686806 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:25.686718    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podUID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 13:36:26.686433 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:26.686396    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dldkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 13:36:26.686845 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:26.686456    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podUID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 13:36:27.686102 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:27.686068    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dldkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 13:36:27.686303 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:27.686127    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podUID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 13:36:28.685934 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:28.685891    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dldkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 13:36:28.686307 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:28.685956    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podUID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 13:36:29.685786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:29.685753    2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-dldkt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Sep 09 13:36:29.685955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:29.685800    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt" podUID="411812a1-de71-41c8-8f62-e0e65a822c8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Sep 09 13:36:30.605418 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:30.605388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-mz5t5"
Sep 09 13:36:30.605640 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:30.605456    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-4s7cw"
Sep 09 13:36:30.689510 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:30.689483    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:36:30.689991 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:30.689970    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:36:30.690740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:30.690717    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-dldkt"
Sep 09 13:36:54.566530 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.566449    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"]
Sep 09 13:36:54.574779 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.574752    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.577467 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.577447    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 13:36:54.578939 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.578484    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ztzlx\""
Sep 09 13:36:54.578939 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.578744    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 13:36:54.578939 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.578761    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 09 13:36:54.583769 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.583746    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"]
Sep 09 13:36:54.666879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.666848    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.666879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.666879    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.667110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.666898    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5jkl\" (UniqueName: \"kubernetes.io/projected/96bcc051-9818-4e81-a1a1-7216424dbb0c-kube-api-access-l5jkl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.667110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.666976    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bcc051-9818-4e81-a1a1-7216424dbb0c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.667110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.667053    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.667110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.667081    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.767859 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.767831    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.767859 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.767861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768029 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.767878    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5jkl\" (UniqueName: \"kubernetes.io/projected/96bcc051-9818-4e81-a1a1-7216424dbb0c-kube-api-access-l5jkl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768029 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768009    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bcc051-9818-4e81-a1a1-7216424dbb0c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768067    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768112 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768095    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768332 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768310    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768417 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768397    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.768464 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.768417    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.770064 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.770044    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/96bcc051-9818-4e81-a1a1-7216424dbb0c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.770415 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.770398    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96bcc051-9818-4e81-a1a1-7216424dbb0c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.776183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.776164    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5jkl\" (UniqueName: \"kubernetes.io/projected/96bcc051-9818-4e81-a1a1-7216424dbb0c-kube-api-access-l5jkl\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6\" (UID: \"96bcc051-9818-4e81-a1a1-7216424dbb0c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:54.888888 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:54.888815    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:36:55.165976 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:55.165949    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"]
Sep 09 13:36:55.167773 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:36:55.167744    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96bcc051_9818_4e81_a1a1_7216424dbb0c.slice/crio-c4e0f120e27d924e075a0c1dcccdf07ac4fd630d32bd7349d416ccad7292833f WatchSource:0}: Error finding container c4e0f120e27d924e075a0c1dcccdf07ac4fd630d32bd7349d416ccad7292833f: Status 404 returned error can't find the container with id c4e0f120e27d924e075a0c1dcccdf07ac4fd630d32bd7349d416ccad7292833f
Sep 09 13:36:55.169164 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:55.169146    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:36:55.807279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:55.807238    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6" event={"ID":"96bcc051-9818-4e81-a1a1-7216424dbb0c","Type":"ContainerStarted","Data":"c4e0f120e27d924e075a0c1dcccdf07ac4fd630d32bd7349d416ccad7292833f"}
Sep 09 13:36:56.815052 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:56.815019    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6" event={"ID":"96bcc051-9818-4e81-a1a1-7216424dbb0c","Type":"ContainerStarted","Data":"220fab1bb12e416ad2e53b67ec01f6a881bacb8ac0764649b2fc1abd5b45e73d"}
Sep 09 13:36:56.833873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:36:56.833827    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6" podStartSLOduration=1.733766602 podStartE2EDuration="2.833812227s" totalImagesPullingTime="1.100045625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:36:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:36:56.832681518 +0000 UTC m=+635.721381519" watchObservedRunningTime="2026-09-09 13:36:56.833812227 +0000 UTC m=+635.722512217"
Sep 09 13:37:04.889068 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:37:04.889033    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:37:04.889431 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:37:04.889091    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:37:04.901202 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:37:04.901179    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:37:05.877170 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:37:05.877140    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6"
Sep 09 13:38:39.391334 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.391244    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-64889fc9d6-fqr9m"]
Sep 09 13:38:39.395139 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.395120    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.403389 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.403367    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64889fc9d6-fqr9m"]
Sep 09 13:38:39.476627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.476598    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f4eb108-effa-4ba9-a918-96ab305ff003-tls-cert\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.476873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.476747    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmpgw\" (UniqueName: \"kubernetes.io/projected/7f4eb108-effa-4ba9-a918-96ab305ff003-kube-api-access-pmpgw\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.577302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.577268    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmpgw\" (UniqueName: \"kubernetes.io/projected/7f4eb108-effa-4ba9-a918-96ab305ff003-kube-api-access-pmpgw\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.577449 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.577318    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f4eb108-effa-4ba9-a918-96ab305ff003-tls-cert\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.579794 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.579770    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7f4eb108-effa-4ba9-a918-96ab305ff003-tls-cert\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.586686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.586666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmpgw\" (UniqueName: \"kubernetes.io/projected/7f4eb108-effa-4ba9-a918-96ab305ff003-kube-api-access-pmpgw\") pod \"authorino-64889fc9d6-fqr9m\" (UID: \"7f4eb108-effa-4ba9-a918-96ab305ff003\") " pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.705493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.705407    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-64889fc9d6-fqr9m"
Sep 09 13:38:39.977879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:39.977852    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-64889fc9d6-fqr9m"]
Sep 09 13:38:39.978931 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:38:39.978902    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f4eb108_effa_4ba9_a918_96ab305ff003.slice/crio-e2528175b80d6d235c5bd649b3b4b66d331d9b21422f946461efa0e933bcd605 WatchSource:0}: Error finding container e2528175b80d6d235c5bd649b3b4b66d331d9b21422f946461efa0e933bcd605: Status 404 returned error can't find the container with id e2528175b80d6d235c5bd649b3b4b66d331d9b21422f946461efa0e933bcd605
Sep 09 13:38:40.419215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:40.419130    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64889fc9d6-fqr9m" event={"ID":"7f4eb108-effa-4ba9-a918-96ab305ff003","Type":"ContainerStarted","Data":"e2528175b80d6d235c5bd649b3b4b66d331d9b21422f946461efa0e933bcd605"}
Sep 09 13:38:41.428077 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.428034    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-64889fc9d6-fqr9m" event={"ID":"7f4eb108-effa-4ba9-a918-96ab305ff003","Type":"ContainerStarted","Data":"13f1f227e4a1c80427a7b4205de27f3884e47953bba9725e8464ab3df7473b7f"}
Sep 09 13:38:41.448357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.448302    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-64889fc9d6-fqr9m" podStartSLOduration=1.607154967 podStartE2EDuration="2.448285371s" totalImagesPullingTime="841.130404ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:38:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:38:41.445095714 +0000 UTC m=+740.333795716" watchObservedRunningTime="2026-09-09 13:38:41.448285371 +0000 UTC m=+740.336985361"
Sep 09 13:38:41.478624 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.478585    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:38:41.478825 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.478800    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" podUID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" containerName="authorino" containerID="cri-o://37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620" gracePeriod=30
Sep 09 13:38:41.727807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.727778    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:38:41.804013 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.803979    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql8z9\" (UniqueName: \"kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9\") pod \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") "
Sep 09 13:38:41.804209 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.804058    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert\") pod \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\" (UID: \"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb\") "
Sep 09 13:38:41.806865 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.806694    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9" (OuterVolumeSpecName: "kube-api-access-ql8z9") pod "b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" (UID: "b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb"). InnerVolumeSpecName "kube-api-access-ql8z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:38:41.815487 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.815459    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" (UID: "b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:38:41.905772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.905725    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql8z9\" (UniqueName: \"kubernetes.io/projected/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-kube-api-access-ql8z9\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:38:41.905772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:41.905767    2559 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb-tls-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:38:42.436069 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.436037    2559 generic.go:356] "Generic (PLEG): container finished" podID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" containerID="37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620" exitCode=0
Sep 09 13:38:42.436490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.436081    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" event={"ID":"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb","Type":"ContainerDied","Data":"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"}
Sep 09 13:38:42.436490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.436142    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd" event={"ID":"b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb","Type":"ContainerDied","Data":"34a26ba7de1dcd80415ed2638e2ae003486e3e2c91eda9dde596628e256d2405"}
Sep 09 13:38:42.436490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.436166    2559 scope.go:117] "RemoveContainer" containerID="37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"
Sep 09 13:38:42.436490 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.436098    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b5dd7b4f5-7cndd"
Sep 09 13:38:42.447046 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.447026    2559 scope.go:117] "RemoveContainer" containerID="37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"
Sep 09 13:38:42.447375 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:38:42.447356    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620\": container with ID starting with 37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620 not found: ID does not exist" containerID="37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"
Sep 09 13:38:42.447449 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.447387    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620"} err="failed to get container status \"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620\": rpc error: code = NotFound desc = could not find container \"37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620\": container with ID starting with 37a82fdf2da0531d2aa23e1d941fcf623f06bcd5cb293afbdea117d30be92620 not found: ID does not exist"
Sep 09 13:38:42.461583 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.461555    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:38:42.464186 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:42.464161    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5b5dd7b4f5-7cndd"]
Sep 09 13:38:43.618100 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:38:43.618056    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" path="/var/lib/kubelet/pods/b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb/volumes"
Sep 09 13:39:01.356602 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.356552    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7949-5d577c"]
Sep 09 13:39:01.357034 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.357019    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" containerName="authorino"
Sep 09 13:39:01.357088 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.357036    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" containerName="authorino"
Sep 09 13:39:01.357124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.357100    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0c4a8b5-232c-48c3-b7aa-7b9ce7d1eccb" containerName="authorino"
Sep 09 13:39:01.359911 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.359895    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:01.362399 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.362381    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qg8jq\""
Sep 09 13:39:01.367924 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.367700    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7949-5d577c"]
Sep 09 13:39:01.472044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.472016    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd8ct\" (UniqueName: \"kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct\") pod \"test-curl-7949-5d577c\" (UID: \"5a658c7f-594b-43d4-8159-d458567ce468\") " pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:01.573342 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.573309    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd8ct\" (UniqueName: \"kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct\") pod \"test-curl-7949-5d577c\" (UID: \"5a658c7f-594b-43d4-8159-d458567ce468\") " pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:01.583694 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.583671    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd8ct\" (UniqueName: \"kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct\") pod \"test-curl-7949-5d577c\" (UID: \"5a658c7f-594b-43d4-8159-d458567ce468\") " pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:01.671051 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.670977    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:01.950466 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:01.950436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7949-5d577c"]
Sep 09 13:39:01.951466 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:01.951439    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a658c7f_594b_43d4_8159_d458567ce468.slice/crio-db04100d0794697536eaef81bdb076732ee73175aacec4228c3033aa59640e68 WatchSource:0}: Error finding container db04100d0794697536eaef81bdb076732ee73175aacec4228c3033aa59640e68: Status 404 returned error can't find the container with id db04100d0794697536eaef81bdb076732ee73175aacec4228c3033aa59640e68
Sep 09 13:39:02.554393 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:02.554357    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7949-5d577c" event={"ID":"5a658c7f-594b-43d4-8159-d458567ce468","Type":"ContainerStarted","Data":"db04100d0794697536eaef81bdb076732ee73175aacec4228c3033aa59640e68"}
Sep 09 13:39:03.619745 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.619712    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:39:03.623919 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.623900    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.626851 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.626779    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4f59a0-serving-cert\""
Sep 09 13:39:03.631352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.631300    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:39:03.795691 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.795662    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.795796 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.795705    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.795840 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.795819    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.795893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.795877    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7z5t\" (UniqueName: \"kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.896515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.896429    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7z5t\" (UniqueName: \"kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.896515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.896480    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.896515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.896500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.896827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.896550    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.897236 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.897216    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.898893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.898867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.898989 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.898912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.904564 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.904544    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7z5t\" (UniqueName: \"kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t\") pod \"maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:03.938501 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:03.938478    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:04.224011 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.223980    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:39:04.224585 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:04.224550    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod148de768_f804_42d1_bd6a_4e1a1f719731.slice/crio-e1d34073258f21843d946e13233d022779c62b94af3e30666895a5e7ca586f5e WatchSource:0}: Error finding container e1d34073258f21843d946e13233d022779c62b94af3e30666895a5e7ca586f5e: Status 404 returned error can't find the container with id e1d34073258f21843d946e13233d022779c62b94af3e30666895a5e7ca586f5e
Sep 09 13:39:04.376937 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.376902    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:39:04.381949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.381931    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.384670 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.384638    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-4f59a0-dockercfg-z9rwp\""
Sep 09 13:39:04.384670 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.384662    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-4f59a0\""
Sep 09 13:39:04.393313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.393281    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:39:04.502536 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.502437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.502701 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.502541    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzd56\" (UniqueName: \"kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.569841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.569794    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7949-5d577c" event={"ID":"5a658c7f-594b-43d4-8159-d458567ce468","Type":"ContainerStarted","Data":"d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532"}
Sep 09 13:39:04.571186 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.571158    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" event={"ID":"148de768-f804-42d1-bd6a-4e1a1f719731","Type":"ContainerStarted","Data":"e1d34073258f21843d946e13233d022779c62b94af3e30666895a5e7ca586f5e"}
Sep 09 13:39:04.588306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.588256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7949-5d577c" podStartSLOduration=1.835578424 podStartE2EDuration="3.588241411s" totalImagesPullingTime="1.752662987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:04.585901008 +0000 UTC m=+763.474600997" watchObservedRunningTime="2026-09-09 13:39:04.588241411 +0000 UTC m=+763.476941447"
Sep 09 13:39:04.603201 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.603179    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.603324 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.603215    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzd56\" (UniqueName: \"kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.604027 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.603997    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.612849 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.612825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzd56\" (UniqueName: \"kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56\") pod \"payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.695728 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.695702    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:04.991044 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:04.991013    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ccdd0a3_0844_4f93_8721_adc220f3238c.slice/crio-5450777a2af1795850be3dfd234d4ca2ad49c6e737cc396a613e0d44884d4a4d WatchSource:0}: Error finding container 5450777a2af1795850be3dfd234d4ca2ad49c6e737cc396a613e0d44884d4a4d: Status 404 returned error can't find the container with id 5450777a2af1795850be3dfd234d4ca2ad49c6e737cc396a613e0d44884d4a4d
Sep 09 13:39:04.991287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:04.991260    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:39:05.198720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.198687    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7949-5d577c"]
Sep 09 13:39:05.202891 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.202863    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7949-5d577c"]
Sep 09 13:39:05.579202 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.579165    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" event={"ID":"148de768-f804-42d1-bd6a-4e1a1f719731","Type":"ContainerStarted","Data":"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"}
Sep 09 13:39:05.579400 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.579252    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:05.580927 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.580903    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" event={"ID":"8ccdd0a3-0844-4f93-8721-adc220f3238c","Type":"ContainerStarted","Data":"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"}
Sep 09 13:39:05.581069 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.580943    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:05.581069 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.580961    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" event={"ID":"8ccdd0a3-0844-4f93-8721-adc220f3238c","Type":"ContainerStarted","Data":"5450777a2af1795850be3dfd234d4ca2ad49c6e737cc396a613e0d44884d4a4d"}
Sep 09 13:39:05.581209 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:05.581191    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7949-5d577c" container="test-curl-7949-5d577c"
Sep 09 13:39:05.596461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.596422    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" podStartSLOduration=2.172647623 podStartE2EDuration="2.596408255s" totalImagesPullingTime="423.760632ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:05.595216446 +0000 UTC m=+764.483916435" watchObservedRunningTime="2026-09-09 13:39:05.596408255 +0000 UTC m=+764.485108245"
Sep 09 13:39:05.620462 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:05.620406    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" podStartSLOduration=1.620386051 podStartE2EDuration="1.620386051s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:05.617052712 +0000 UTC m=+764.505752704" watchObservedRunningTime="2026-09-09 13:39:05.620386051 +0000 UTC m=+764.509086042"
Sep 09 13:39:06.587648 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:06.587565    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7949-5d577c" podUID="5a658c7f-594b-43d4-8159-d458567ce468" containerName="test-curl-7949-5d577c" containerID="cri-o://d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532" gracePeriod=2
Sep 09 13:39:08.808601 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:08.808540    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:08.947916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:08.947813    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd8ct\" (UniqueName: \"kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct\") pod \"5a658c7f-594b-43d4-8159-d458567ce468\" (UID: \"5a658c7f-594b-43d4-8159-d458567ce468\") "
Sep 09 13:39:08.950023 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:08.949989    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct" (OuterVolumeSpecName: "kube-api-access-fd8ct") pod "5a658c7f-594b-43d4-8159-d458567ce468" (UID: "5a658c7f-594b-43d4-8159-d458567ce468"). InnerVolumeSpecName "kube-api-access-fd8ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:39:09.048846 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.048812    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd8ct\" (UniqueName: \"kubernetes.io/projected/5a658c7f-594b-43d4-8159-d458567ce468-kube-api-access-fd8ct\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:39:09.608758 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.608721    2559 generic.go:356] "Generic (PLEG): container finished" podID="5a658c7f-594b-43d4-8159-d458567ce468" containerID="d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532" exitCode=137
Sep 09 13:39:09.608984 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.608824    2559 scope.go:117] "RemoveContainer" containerID="d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532"
Sep 09 13:39:09.608984 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.608830    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:09.629746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.629711    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a658c7f-594b-43d4-8159-d458567ce468" path="/var/lib/kubelet/pods/5a658c7f-594b-43d4-8159-d458567ce468/volumes"
Sep 09 13:39:09.631690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.631159    2559 scope.go:117] "RemoveContainer" containerID="d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532"
Sep 09 13:39:09.631690 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:09.631244    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7949-5d577c\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="5a658c7f-594b-43d4-8159-d458567ce468" pod="opendatahub/test-curl-7949-5d577c"
Sep 09 13:39:09.634588 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:09.633923    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532\": container with ID starting with d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532 not found: ID does not exist" containerID="d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532"
Sep 09 13:39:09.634588 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:09.633961    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532"} err="failed to get container status \"d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532\": rpc error: code = NotFound desc = could not find container \"d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532\": container with ID starting with d8696bed2e4b7adfe5b7d358a79b07789307d38545e937cc90bddd04b111c532 not found: ID does not exist"
Sep 09 13:39:11.592908 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:11.592882    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:39:12.837876 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.837838    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:12.838343 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.838329    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a658c7f-594b-43d4-8159-d458567ce468" containerName="test-curl-7949-5d577c"
Sep 09 13:39:12.838388 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.838344    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a658c7f-594b-43d4-8159-d458567ce468" containerName="test-curl-7949-5d577c"
Sep 09 13:39:12.838428 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.838417    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a658c7f-594b-43d4-8159-d458567ce468" containerName="test-curl-7949-5d577c"
Sep 09 13:39:12.842956 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.842931    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.845902 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.845881    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a649773d\""
Sep 09 13:39:12.855142 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.855119    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:12.887298 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.887263    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ds6\" (UniqueName: \"kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.887482 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.887328    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.960990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.960953    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:12.964851 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.964830    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:12.967496 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.967473    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a649773d-dockercfg-sjgc8\""
Sep 09 13:39:12.978662 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.978633    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:12.988024 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.987999    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ds6\" (UniqueName: \"kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.988136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.988043    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6t6b\" (UniqueName: \"kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:12.988136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.988089    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.988253 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.988211    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:12.988808 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.988790    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:12.997895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:12.997872    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ds6\" (UniqueName: \"kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6\") pod \"payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:13.089272 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.089183    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:13.089444 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.089391    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6t6b\" (UniqueName: \"kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:13.089963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.089943    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:13.104990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.104960    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6t6b\" (UniqueName: \"kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b\") pod \"payload-processing-e2e-ait-a649773d-5db8b5677c-kj498\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") " pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:13.153492 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.153456    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:13.277449 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.277416    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:13.435840 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.435808    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:13.436978 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:13.436941    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f78f36f_eb22_4552_bdc4_2f0aa5ebaace.slice/crio-f47565ef1bc8d4d5cd65897367c0bfaf4b4f759167869ae4ac146efead7e6286 WatchSource:0}: Error finding container f47565ef1bc8d4d5cd65897367c0bfaf4b4f759167869ae4ac146efead7e6286: Status 404 returned error can't find the container with id f47565ef1bc8d4d5cd65897367c0bfaf4b4f759167869ae4ac146efead7e6286
Sep 09 13:39:13.591980 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:13.591949    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda66d869c_6bf9_4f35_af57_32d2d6e75ef0.slice/crio-86985b185c075d18983af1573310bef109d7c18ae155f0e8a953a084ab96f24e WatchSource:0}: Error finding container 86985b185c075d18983af1573310bef109d7c18ae155f0e8a953a084ab96f24e: Status 404 returned error can't find the container with id 86985b185c075d18983af1573310bef109d7c18ae155f0e8a953a084ab96f24e
Sep 09 13:39:13.594498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.594474    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:13.638612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.638566    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" event={"ID":"a66d869c-6bf9-4f35-af57-32d2d6e75ef0","Type":"ContainerStarted","Data":"86985b185c075d18983af1573310bef109d7c18ae155f0e8a953a084ab96f24e"}
Sep 09 13:39:13.640328 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.640302    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" event={"ID":"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace","Type":"ContainerStarted","Data":"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"}
Sep 09 13:39:13.640431 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.640347    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" event={"ID":"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace","Type":"ContainerStarted","Data":"f47565ef1bc8d4d5cd65897367c0bfaf4b4f759167869ae4ac146efead7e6286"}
Sep 09 13:39:13.640531 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.640511    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:13.672193 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:13.672141    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" podStartSLOduration=1.6721241980000001 podStartE2EDuration="1.672124198s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:13.669387152 +0000 UTC m=+772.558087142" watchObservedRunningTime="2026-09-09 13:39:13.672124198 +0000 UTC m=+772.560824187"
Sep 09 13:39:14.649178 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.649141    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" event={"ID":"a66d869c-6bf9-4f35-af57-32d2d6e75ef0","Type":"ContainerStarted","Data":"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"}
Sep 09 13:39:14.649650 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.649251    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:14.687994 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.687945    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" podStartSLOduration=2.687930811 podStartE2EDuration="2.687930811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:14.684285398 +0000 UTC m=+773.572985387" watchObservedRunningTime="2026-09-09 13:39:14.687930811 +0000 UTC m=+773.576630801"
Sep 09 13:39:14.990406 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.990369    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:39:14.994280 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.994254    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:14.996994 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:14.996967    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-71c181-serving-cert\""
Sep 09 13:39:15.011828 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.011726    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:39:15.108520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.108472    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.108724 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.108635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.108790 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.108715    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzzwl\" (UniqueName: \"kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.108829 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.108800    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.209784 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.209743    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.209974 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.209833    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.209974 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.209863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzzwl\" (UniqueName: \"kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.209974 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.209890    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.209974 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:15.209904    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-71c181-serving-cert: secret "maas-api-e2e-autores-71c181-serving-cert" not found
Sep 09 13:39:15.209974 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:15.209971    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls podName:b4770d43-1788-497d-acb9-916ad9369ce2 nodeName:}" failed. No retries permitted until 2026-09-09 13:39:15.709949881 +0000 UTC m=+774.598649872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls") pod "maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" (UID: "b4770d43-1788-497d-acb9-916ad9369ce2") : secret "maas-api-e2e-autores-71c181-serving-cert" not found
Sep 09 13:39:15.210695 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.210666    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.212378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.212354    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.243098 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.243026    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzzwl\" (UniqueName: \"kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.714282 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.714249    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.716696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.716668    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") pod \"maas-api-e2e-autores-71c181-6dccb77fcb-btdtf\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:15.908677 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:15.908638    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:16.223335 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:16.223306    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4770d43_1788_497d_acb9_916ad9369ce2.slice/crio-32de08419a5fc123d213d2ee1c0e6baf16e7e5ef492e13ff660bab589dc5bd45 WatchSource:0}: Error finding container 32de08419a5fc123d213d2ee1c0e6baf16e7e5ef492e13ff660bab589dc5bd45: Status 404 returned error can't find the container with id 32de08419a5fc123d213d2ee1c0e6baf16e7e5ef492e13ff660bab589dc5bd45
Sep 09 13:39:16.223817 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.223791    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:39:16.590486 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.590458    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:39:16.663983 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.663935    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" event={"ID":"b4770d43-1788-497d-acb9-916ad9369ce2","Type":"ContainerStarted","Data":"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"}
Sep 09 13:39:16.664182 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.663995    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" event={"ID":"b4770d43-1788-497d-acb9-916ad9369ce2","Type":"ContainerStarted","Data":"32de08419a5fc123d213d2ee1c0e6baf16e7e5ef492e13ff660bab589dc5bd45"}
Sep 09 13:39:16.664182 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.664066    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:16.700017 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:16.699948    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" podStartSLOduration=2.3772352039999998 podStartE2EDuration="2.699931485s" totalImagesPullingTime="322.696281ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:16.696709593 +0000 UTC m=+775.585409594" watchObservedRunningTime="2026-09-09 13:39:16.699931485 +0000 UTC m=+775.588631474"
Sep 09 13:39:22.676244 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:22.676215    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:39:24.651390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:24.651359    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:25.656967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:25.656936    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:34.044645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.044607    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:39:34.049025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.049003    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.054374 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.054350    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-093e38\""
Sep 09 13:39:34.054983 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.054968    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-093e38-dockercfg-t5k55\""
Sep 09 13:39:34.077335 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.077306    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:39:34.197511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.197469    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.197678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.197593    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.298680 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.298566    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.298857 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.298685    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.299315 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.299290    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.315307 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.315277    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq\") pod \"payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.359272 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.359237    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.642848 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.642815    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:39:34.644089 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:34.644062    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b9befd5_55b6_40c6_94b3_03564d6467ae.slice/crio-b68b66bb54da675424e508cd21ba40f8cc68bc2cc2e370f4ebd141879a1a9b1e WatchSource:0}: Error finding container b68b66bb54da675424e508cd21ba40f8cc68bc2cc2e370f4ebd141879a1a9b1e: Status 404 returned error can't find the container with id b68b66bb54da675424e508cd21ba40f8cc68bc2cc2e370f4ebd141879a1a9b1e
Sep 09 13:39:34.788142 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.788103    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" event={"ID":"7b9befd5-55b6-40c6-94b3-03564d6467ae","Type":"ContainerStarted","Data":"8b8666a758267cf04b33ecc540c9cdd87a9549d0bf5a236a5cc4343e3a4e045c"}
Sep 09 13:39:34.788294 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.788162    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" event={"ID":"7b9befd5-55b6-40c6-94b3-03564d6467ae","Type":"ContainerStarted","Data":"b68b66bb54da675424e508cd21ba40f8cc68bc2cc2e370f4ebd141879a1a9b1e"}
Sep 09 13:39:34.788294 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.788215    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:34.830528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:34.830448    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" podStartSLOduration=1.830424099 podStartE2EDuration="1.830424099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:34.824807243 +0000 UTC m=+793.713507247" watchObservedRunningTime="2026-09-09 13:39:34.830424099 +0000 UTC m=+793.719124092"
Sep 09 13:39:45.797080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:45.797009    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:39:52.133182 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.133151    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:52.133792 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.133762    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" podUID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" containerName="payload-processing" containerID="cri-o://9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91" gracePeriod=30
Sep 09 13:39:52.144403 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.144374    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:52.144693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.144639    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" podUID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" containerName="payload-pre-processing" containerID="cri-o://f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94" gracePeriod=30
Sep 09 13:39:52.434535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.434505    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:52.438713 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.438680    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:52.474687 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.474656    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume\") pod \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") "
Sep 09 13:39:52.474865 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.474703    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume\") pod \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") "
Sep 09 13:39:52.474865 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.474800    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7ds6\" (UniqueName: \"kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6\") pod \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\" (UID: \"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace\") "
Sep 09 13:39:52.474865 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.474827    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n6t6b\" (UniqueName: \"kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b\") pod \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\" (UID: \"a66d869c-6bf9-4f35-af57-32d2d6e75ef0\") "
Sep 09 13:39:52.475835 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.475682    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" (UID: "1f78f36f-eb22-4552-bdc4-2f0aa5ebaace"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:39:52.476026 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.476001    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a66d869c-6bf9-4f35-af57-32d2d6e75ef0" (UID: "a66d869c-6bf9-4f35-af57-32d2d6e75ef0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:39:52.478403 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.478375    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6" (OuterVolumeSpecName: "kube-api-access-p7ds6") pod "1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" (UID: "1f78f36f-eb22-4552-bdc4-2f0aa5ebaace"). InnerVolumeSpecName "kube-api-access-p7ds6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:39:52.478897 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.478867    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b" (OuterVolumeSpecName: "kube-api-access-n6t6b") pod "a66d869c-6bf9-4f35-af57-32d2d6e75ef0" (UID: "a66d869c-6bf9-4f35-af57-32d2d6e75ef0"). InnerVolumeSpecName "kube-api-access-n6t6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:39:52.575685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.575650    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-plugins-config-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:39:52.575685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.575679    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-plugins-config-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:39:52.575685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.575689    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7ds6\" (UniqueName: \"kubernetes.io/projected/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace-kube-api-access-p7ds6\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:39:52.575909 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.575698    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n6t6b\" (UniqueName: \"kubernetes.io/projected/a66d869c-6bf9-4f35-af57-32d2d6e75ef0-kube-api-access-n6t6b\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:39:52.901185 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.901148    2559 generic.go:356] "Generic (PLEG): container finished" podID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" containerID="f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94" exitCode=0
Sep 09 13:39:52.901346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.901213    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"
Sep 09 13:39:52.901346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.901235    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" event={"ID":"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace","Type":"ContainerDied","Data":"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"}
Sep 09 13:39:52.901346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.901290    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp" event={"ID":"1f78f36f-eb22-4552-bdc4-2f0aa5ebaace","Type":"ContainerDied","Data":"f47565ef1bc8d4d5cd65897367c0bfaf4b4f759167869ae4ac146efead7e6286"}
Sep 09 13:39:52.901346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.901320    2559 scope.go:117] "RemoveContainer" containerID="f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"
Sep 09 13:39:52.902652 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.902628    2559 generic.go:356] "Generic (PLEG): container finished" podID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" containerID="9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91" exitCode=0
Sep 09 13:39:52.902772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.902682    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"
Sep 09 13:39:52.902772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.902687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" event={"ID":"a66d869c-6bf9-4f35-af57-32d2d6e75ef0","Type":"ContainerDied","Data":"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"}
Sep 09 13:39:52.902772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.902733    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498" event={"ID":"a66d869c-6bf9-4f35-af57-32d2d6e75ef0","Type":"ContainerDied","Data":"86985b185c075d18983af1573310bef109d7c18ae155f0e8a953a084ab96f24e"}
Sep 09 13:39:52.912953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.912899    2559 scope.go:117] "RemoveContainer" containerID="f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"
Sep 09 13:39:52.913545 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:52.913393    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94\": container with ID starting with f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94 not found: ID does not exist" containerID="f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"
Sep 09 13:39:52.913545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.913431    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94"} err="failed to get container status \"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94\": rpc error: code = NotFound desc = could not find container \"f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94\": container with ID starting with f96704f42e7949e8bc1bcee087b1cbd2dce87d953ff937cf4288bbc887eaec94 not found: ID does not exist"
Sep 09 13:39:52.913545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.913457    2559 scope.go:117] "RemoveContainer" containerID="9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"
Sep 09 13:39:52.924127 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.924111    2559 scope.go:117] "RemoveContainer" containerID="9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"
Sep 09 13:39:52.924385 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:39:52.924365    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91\": container with ID starting with 9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91 not found: ID does not exist" containerID="9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"
Sep 09 13:39:52.924438 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.924392    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91"} err="failed to get container status \"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91\": rpc error: code = NotFound desc = could not find container \"9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91\": container with ID starting with 9bef3cc38dfb68bcbc607264e03595696093b35d855a78f2e600eb9889ae5f91 not found: ID does not exist"
Sep 09 13:39:52.948857 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.948836    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:52.966800 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:52.966776    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a649773d-6fbc965d7d-jv7gp"]
Sep 09 13:39:53.011161 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.011136    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:53.022072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.022047    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-a649773d-5db8b5677c-kj498"]
Sep 09 13:39:53.433918 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.433843    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:39:53.434586 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434547    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" containerName="payload-processing"
Sep 09 13:39:53.434692 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434589    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" containerName="payload-processing"
Sep 09 13:39:53.434692 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434631    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" containerName="payload-pre-processing"
Sep 09 13:39:53.434692 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434640    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" containerName="payload-pre-processing"
Sep 09 13:39:53.434854 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434776    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" containerName="payload-pre-processing"
Sep 09 13:39:53.434854 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.434789    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" containerName="payload-processing"
Sep 09 13:39:53.439659 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.439638    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.442306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.442286    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-219dfd41-gw-openshift-default-dockercfg-5gqtm\""
Sep 09 13:39:53.450275 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.450242    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:39:53.483117 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483085    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483138    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483323 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483270    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483323 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483409 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483417    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbz8m\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483448    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.483563 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.483475    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584031 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.583991    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584042    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584093    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584129    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584225 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584169    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584502 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584502 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584280    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbz8m\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584502 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584502 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584333    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584624    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584663    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.584931 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.584905    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.585120 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.585096    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.585329 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.585303    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.586801 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.586778    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.587001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.586983    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.596189 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.596166    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.600666 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.600646    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbz8m\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m\") pod \"e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") " pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:53.618877 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.618852    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f78f36f-eb22-4552-bdc4-2f0aa5ebaace" path="/var/lib/kubelet/pods/1f78f36f-eb22-4552-bdc4-2f0aa5ebaace/volumes"
Sep 09 13:39:53.619269 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.619254    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a66d869c-6bf9-4f35-af57-32d2d6e75ef0" path="/var/lib/kubelet/pods/a66d869c-6bf9-4f35-af57-32d2d6e75ef0/volumes"
Sep 09 13:39:53.752307 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:53.752285    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:54.039900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.039590    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:39:54.041837 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:39:54.041804    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod267144b1_f61b_4cf1_bac9_b3739fd9d350.slice/crio-f2d107662809a02c0de2f9c3242efb47a0d69e606ad43ed1e77aff87a773334c WatchSource:0}: Error finding container f2d107662809a02c0de2f9c3242efb47a0d69e606ad43ed1e77aff87a773334c: Status 404 returned error can't find the container with id f2d107662809a02c0de2f9c3242efb47a0d69e606ad43ed1e77aff87a773334c
Sep 09 13:39:54.044184 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.044154    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:39:54.044249 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.044215    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:39:54.044287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.044247    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:39:54.921086 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.921053    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerStarted","Data":"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"}
Sep 09 13:39:54.921465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.921109    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerStarted","Data":"f2d107662809a02c0de2f9c3242efb47a0d69e606ad43ed1e77aff87a773334c"}
Sep 09 13:39:54.945822 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:54.945776    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podStartSLOduration=1.9457621619999999 podStartE2EDuration="1.945762162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:39:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:39:54.942698291 +0000 UTC m=+813.831398281" watchObservedRunningTime="2026-09-09 13:39:54.945762162 +0000 UTC m=+813.834462151"
Sep 09 13:39:55.752642 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:55.752605    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:55.758114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:55.758089    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:55.927371 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:55.927341    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:39:55.928295 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:39:55.928272    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:19.087545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.087475    2559 generic.go:356] "Generic (PLEG): container finished" podID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerID="86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d" exitCode=0
Sep 09 13:40:19.087545 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.087516    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerDied","Data":"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"}
Sep 09 13:40:19.087996 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.087912    2559 scope.go:117] "RemoveContainer" containerID="86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"
Sep 09 13:40:19.089040 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.088998    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:40:19.089101 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.089086    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:40:19.089136 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:19.089119    2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 09 13:40:20.095154 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:20.095114    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerStarted","Data":"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"}
Sep 09 13:40:20.753075 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:20.753044    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:20.753287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:20.753265    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:20.753333 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:20.753315    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:21.752856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:21.752813    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:21.753274 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:21.752876    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:22.752610 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:22.752561    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:22.752824 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:22.752649    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:23.752666 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:23.752622    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:23.753158 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:23.752701    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:24.753596 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:24.753537    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:24.754048 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:24.753617    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:25.753313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:25.753273    2559 patch_prober.go:28] interesting pod/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused" start-of-body=
Sep 09 13:40:25.753491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:25.753336    2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.69:15021/healthz/ready\": dial tcp 10.133.0.69:15021: connect: connection refused"
Sep 09 13:40:25.928073 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:25.928041    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:26.756893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:26.756860    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:26.757918 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:26.757893    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:28.415188 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:28.415153    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:40:28.415933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:28.415899    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy" containerID="cri-o://d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d" gracePeriod=30
Sep 09 13:40:33.783316 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.783291    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:33.785599 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785566    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785612    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785638    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785661    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785692    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785730    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785754    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.785856 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785804    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.786065 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785857    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbz8m\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m\") pod \"267144b1-f61b-4cf1-bac9-b3739fd9d350\" (UID: \"267144b1-f61b-4cf1-bac9-b3739fd9d350\") "
Sep 09 13:40:33.786065 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785862    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:40:33.786065 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.785975    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:40:33.786222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786106    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data" (OuterVolumeSpecName: "istio-data") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:40:33.786222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786152    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:40:33.786222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786209    2559 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-data\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.786379 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786230    2559 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.786379 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786245    2559 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-credential-socket\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.786467 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.786374    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:40:33.788150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.788119    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m" (OuterVolumeSpecName: "kube-api-access-mbz8m") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "kube-api-access-mbz8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:40:33.788619 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.788595    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token" (OuterVolumeSpecName: "istio-token") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:40:33.788619 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.788610    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:40:33.788774 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.788623    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "267144b1-f61b-4cf1-bac9-b3739fd9d350" (UID: "267144b1-f61b-4cf1-bac9-b3739fd9d350"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:40:33.886756 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886729    2559 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-envoy\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.886756 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886754    2559 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/267144b1-f61b-4cf1-bac9-b3739fd9d350-istiod-ca-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.886756 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886763    2559 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-podinfo\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.886967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886771    2559 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-istio-token\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.886967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886778    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbz8m\" (UniqueName: \"kubernetes.io/projected/267144b1-f61b-4cf1-bac9-b3739fd9d350-kube-api-access-mbz8m\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:33.886967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:33.886788    2559 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/267144b1-f61b-4cf1-bac9-b3739fd9d350-workload-socket\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:40:34.198818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.198738    2559 generic.go:356] "Generic (PLEG): container finished" podID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerID="d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d" exitCode=0
Sep 09 13:40:34.198818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.198784    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"
Sep 09 13:40:34.198818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.198792    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerDied","Data":"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"}
Sep 09 13:40:34.199051 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.198843    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss" event={"ID":"267144b1-f61b-4cf1-bac9-b3739fd9d350","Type":"ContainerDied","Data":"f2d107662809a02c0de2f9c3242efb47a0d69e606ad43ed1e77aff87a773334c"}
Sep 09 13:40:34.199051 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.198868    2559 scope.go:117] "RemoveContainer" containerID="d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"
Sep 09 13:40:34.209539 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.209515    2559 scope.go:117] "RemoveContainer" containerID="86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"
Sep 09 13:40:34.218119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.218096    2559 scope.go:117] "RemoveContainer" containerID="d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"
Sep 09 13:40:34.218379 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:40:34.218361    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d\": container with ID starting with d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d not found: ID does not exist" containerID="d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"
Sep 09 13:40:34.218463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.218384    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d"} err="failed to get container status \"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d\": rpc error: code = NotFound desc = could not find container \"d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d\": container with ID starting with d00c4df4e1ac1923d79bdaca0715aa3d52b659693cf74d06a8a4c748d5b6a17d not found: ID does not exist"
Sep 09 13:40:34.218463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.218400    2559 scope.go:117] "RemoveContainer" containerID="86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"
Sep 09 13:40:34.218688 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:40:34.218667    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d\": container with ID starting with 86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d not found: ID does not exist" containerID="86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"
Sep 09 13:40:34.218769 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.218692    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d"} err="failed to get container status \"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d\": rpc error: code = NotFound desc = could not find container \"86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d\": container with ID starting with 86eb3e942498cd75ab02a95196ae0975db77d35629464beb3dcf9aac03275e2d not found: ID does not exist"
Sep 09 13:40:34.225175 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.225152    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:40:34.228779 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:34.228758    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-219dfd41-gw-openshift-default-c647446d8-hsgss"]
Sep 09 13:40:35.621811 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:40:35.621777    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" path="/var/lib/kubelet/pods/267144b1-f61b-4cf1-bac9-b3739fd9d350/volumes"
Sep 09 13:41:12.003184 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003147    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:12.003726 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003706    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.003821 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003728    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.003821 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003760    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.003821 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003769    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.003979 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003882    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.003979 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.003896    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="267144b1-f61b-4cf1-bac9-b3739fd9d350" containerName="istio-proxy"
Sep 09 13:41:12.007717 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.007691    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.010500 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.010477    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-4ecd209d-serving-cert\""
Sep 09 13:41:12.019993 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.019966    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:12.147346 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.147308    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.147504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.147418    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.147504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.147444    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.147612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.147507    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgfpd\" (UniqueName: \"kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.248478 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.248434    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.248478 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.248484    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.248763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.248513    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgfpd\" (UniqueName: \"kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.248763 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.248632    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.249397 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.249364    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.250933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.250911    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.251043 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.250957    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.256614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.256539    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgfpd\" (UniqueName: \"kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd\") pod \"maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.344302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.344263    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:12.624732 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:41:12.624555    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod491a24fe_63cf_4aeb_86aa_6f460471bf4e.slice/crio-d8ab052a3d7f747d2ac986ab82ee027cdf401d3bed14fb002bf330b3e15acdfb WatchSource:0}: Error finding container d8ab052a3d7f747d2ac986ab82ee027cdf401d3bed14fb002bf330b3e15acdfb: Status 404 returned error can't find the container with id d8ab052a3d7f747d2ac986ab82ee027cdf401d3bed14fb002bf330b3e15acdfb
Sep 09 13:41:12.625217 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:12.625189    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:13.470045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:13.470006    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" event={"ID":"491a24fe-63cf-4aeb-86aa-6f460471bf4e","Type":"ContainerStarted","Data":"d8ab052a3d7f747d2ac986ab82ee027cdf401d3bed14fb002bf330b3e15acdfb"}
Sep 09 13:41:14.478463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:14.478419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" event={"ID":"491a24fe-63cf-4aeb-86aa-6f460471bf4e","Type":"ContainerStarted","Data":"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"}
Sep 09 13:41:14.478847 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:14.478486    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:14.497836 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:14.497788    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" podStartSLOduration=1.732726015 podStartE2EDuration="3.497772741s" totalImagesPullingTime="1.765046726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:41:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:41:14.496106655 +0000 UTC m=+893.384806646" watchObservedRunningTime="2026-09-09 13:41:14.497772741 +0000 UTC m=+893.386472731"
Sep 09 13:41:20.489965 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:20.489936    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:27.833727 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:27.833695    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:41:27.834171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:27.833939    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" podUID="b4770d43-1788-497d-acb9-916ad9369ce2" containerName="maas-api" containerID="cri-o://ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c" gracePeriod=30
Sep 09 13:41:28.091504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.091430    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:41:28.203645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.203612    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs\") pod \"b4770d43-1788-497d-acb9-916ad9369ce2\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") "
Sep 09 13:41:28.203645 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.203649    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca\") pod \"b4770d43-1788-497d-acb9-916ad9369ce2\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") "
Sep 09 13:41:28.203903 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.203695    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzzwl\" (UniqueName: \"kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl\") pod \"b4770d43-1788-497d-acb9-916ad9369ce2\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") "
Sep 09 13:41:28.203903 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.203792    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") pod \"b4770d43-1788-497d-acb9-916ad9369ce2\" (UID: \"b4770d43-1788-497d-acb9-916ad9369ce2\") "
Sep 09 13:41:28.204384 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.204323    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4770d43-1788-497d-acb9-916ad9369ce2" (UID: "b4770d43-1788-497d-acb9-916ad9369ce2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:41:28.206689 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.206663    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b4770d43-1788-497d-acb9-916ad9369ce2" (UID: "b4770d43-1788-497d-acb9-916ad9369ce2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:41:28.206796 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.206683    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4770d43-1788-497d-acb9-916ad9369ce2" (UID: "b4770d43-1788-497d-acb9-916ad9369ce2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:41:28.206840 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.206796    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl" (OuterVolumeSpecName: "kube-api-access-vzzwl") pod "b4770d43-1788-497d-acb9-916ad9369ce2" (UID: "b4770d43-1788-497d-acb9-916ad9369ce2"). InnerVolumeSpecName "kube-api-access-vzzwl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:41:28.305529 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.305497    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzzwl\" (UniqueName: \"kubernetes.io/projected/b4770d43-1788-497d-acb9-916ad9369ce2-kube-api-access-vzzwl\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:28.305529 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.305525    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:28.305529 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.305537    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b4770d43-1788-497d-acb9-916ad9369ce2-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:28.305798 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.305548    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4770d43-1788-497d-acb9-916ad9369ce2-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:28.578542 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.578504    2559 generic.go:356] "Generic (PLEG): container finished" podID="b4770d43-1788-497d-acb9-916ad9369ce2" containerID="ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c" exitCode=0
Sep 09 13:41:28.578723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.578549    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" event={"ID":"b4770d43-1788-497d-acb9-916ad9369ce2","Type":"ContainerDied","Data":"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"}
Sep 09 13:41:28.578723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.578554    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"
Sep 09 13:41:28.578723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.578618    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf" event={"ID":"b4770d43-1788-497d-acb9-916ad9369ce2","Type":"ContainerDied","Data":"32de08419a5fc123d213d2ee1c0e6baf16e7e5ef492e13ff660bab589dc5bd45"}
Sep 09 13:41:28.578723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.578641    2559 scope.go:117] "RemoveContainer" containerID="ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"
Sep 09 13:41:28.588318 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.588297    2559 scope.go:117] "RemoveContainer" containerID="ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"
Sep 09 13:41:28.588633 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:41:28.588613    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c\": container with ID starting with ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c not found: ID does not exist" containerID="ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"
Sep 09 13:41:28.588696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.588643    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c"} err="failed to get container status \"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c\": rpc error: code = NotFound desc = could not find container \"ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c\": container with ID starting with ae1dfabfebb63ef09a651a8c50504fddbad5e4ddb4283df02f56c501e8bcdf1c not found: ID does not exist"
Sep 09 13:41:28.607584 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.607543    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:41:28.614936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:28.614910    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-71c181-6dccb77fcb-btdtf"]
Sep 09 13:41:29.618258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:29.618225    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4770d43-1788-497d-acb9-916ad9369ce2" path="/var/lib/kubelet/pods/b4770d43-1788-497d-acb9-916ad9369ce2/volumes"
Sep 09 13:41:42.878119 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:42.878083    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:42.878722 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:42.878326    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" podUID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" containerName="maas-api" containerID="cri-o://085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854" gracePeriod=30
Sep 09 13:41:43.136935 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.136882    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:43.243072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.242505    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs\") pod \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") "
Sep 09 13:41:43.243072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.242561    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls\") pod \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") "
Sep 09 13:41:43.243072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.242669    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgfpd\" (UniqueName: \"kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd\") pod \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") "
Sep 09 13:41:43.243072 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.242715    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca\") pod \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\" (UID: \"491a24fe-63cf-4aeb-86aa-6f460471bf4e\") "
Sep 09 13:41:43.243656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.243404    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "491a24fe-63cf-4aeb-86aa-6f460471bf4e" (UID: "491a24fe-63cf-4aeb-86aa-6f460471bf4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:41:43.248429 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.248315    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "491a24fe-63cf-4aeb-86aa-6f460471bf4e" (UID: "491a24fe-63cf-4aeb-86aa-6f460471bf4e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:41:43.249255 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.249230    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd" (OuterVolumeSpecName: "kube-api-access-xgfpd") pod "491a24fe-63cf-4aeb-86aa-6f460471bf4e" (UID: "491a24fe-63cf-4aeb-86aa-6f460471bf4e"). InnerVolumeSpecName "kube-api-access-xgfpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:41:43.250805 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.250774    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "491a24fe-63cf-4aeb-86aa-6f460471bf4e" (UID: "491a24fe-63cf-4aeb-86aa-6f460471bf4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:41:43.344355 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.344322    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:43.344355 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.344350    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/491a24fe-63cf-4aeb-86aa-6f460471bf4e-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:43.344563 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.344364    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgfpd\" (UniqueName: \"kubernetes.io/projected/491a24fe-63cf-4aeb-86aa-6f460471bf4e-kube-api-access-xgfpd\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:43.344563 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.344377    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/491a24fe-63cf-4aeb-86aa-6f460471bf4e-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:41:43.680550 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.680519    2559 generic.go:356] "Generic (PLEG): container finished" podID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" containerID="085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854" exitCode=0
Sep 09 13:41:43.680721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.680560    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"
Sep 09 13:41:43.680721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.680610    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" event={"ID":"491a24fe-63cf-4aeb-86aa-6f460471bf4e","Type":"ContainerDied","Data":"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"}
Sep 09 13:41:43.680721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.680663    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv" event={"ID":"491a24fe-63cf-4aeb-86aa-6f460471bf4e","Type":"ContainerDied","Data":"d8ab052a3d7f747d2ac986ab82ee027cdf401d3bed14fb002bf330b3e15acdfb"}
Sep 09 13:41:43.680721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.680689    2559 scope.go:117] "RemoveContainer" containerID="085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"
Sep 09 13:41:43.689993 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.689982    2559 scope.go:117] "RemoveContainer" containerID="085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"
Sep 09 13:41:43.690281 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:41:43.690261    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854\": container with ID starting with 085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854 not found: ID does not exist" containerID="085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"
Sep 09 13:41:43.690340 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.690291    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854"} err="failed to get container status \"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854\": rpc error: code = NotFound desc = could not find container \"085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854\": container with ID starting with 085b2602a9ff23943e87f3de3da7582b3937e6f91380d49c6ec4deb7c9256854 not found: ID does not exist"
Sep 09 13:41:43.699865 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.699844    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:43.707739 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:43.707714    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-4ecd209d-7666bbf474-vcxnv"]
Sep 09 13:41:45.624931 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:41:45.624894    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" path="/var/lib/kubelet/pods/491a24fe-63cf-4aeb-86aa-6f460471bf4e/volumes"
Sep 09 13:42:15.926260 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.926189    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.926893    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" containerName="maas-api"
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.926912    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" containerName="maas-api"
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.926949    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4770d43-1788-497d-acb9-916ad9369ce2" containerName="maas-api"
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.926958    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4770d43-1788-497d-acb9-916ad9369ce2" containerName="maas-api"
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.927063    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4770d43-1788-497d-acb9-916ad9369ce2" containerName="maas-api"
Sep 09 13:42:15.927187 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.927079    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="491a24fe-63cf-4aeb-86aa-6f460471bf4e" containerName="maas-api"
Sep 09 13:42:15.930864 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.930842    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:15.933638 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.933617    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-c5f375-serving-cert\""
Sep 09 13:42:15.942370 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:15.941731    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:42:16.042516 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.042467    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.042730 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.042635    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.042730 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.042710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.042848 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.042748    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v5dx\" (UniqueName: \"kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.144106 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.144065    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.144283 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.144133    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.144283 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.144166    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v5dx\" (UniqueName: \"kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.144283 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.144212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.144587 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:42:16.144547    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-serving-cert: secret "maas-api-e2e-explicit-c5f375-serving-cert" not found
Sep 09 13:42:16.144701 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:42:16.144638    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls podName:5ed68bc7-aa6b-4710-b598-295470b764a7 nodeName:}" failed. No retries permitted until 2026-09-09 13:42:16.644618536 +0000 UTC m=+955.533318522 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls") pod "maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" (UID: "5ed68bc7-aa6b-4710-b598-295470b764a7") : secret "maas-api-e2e-explicit-c5f375-serving-cert" not found
Sep 09 13:42:16.144890 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.144867    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.147140 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.147115    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.154504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.154483    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v5dx\" (UniqueName: \"kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.649810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.649773    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.652242 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.652211    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") pod \"maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:16.845803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:16.845765    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:17.124361 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:17.124329    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:42:17.125181 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:42:17.125155    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ed68bc7_aa6b_4710_b598_295470b764a7.slice/crio-622bfc6e98cdb1a3d08922c535239ca1913859b2e713ee2437327fa6be59b022 WatchSource:0}: Error finding container 622bfc6e98cdb1a3d08922c535239ca1913859b2e713ee2437327fa6be59b022: Status 404 returned error can't find the container with id 622bfc6e98cdb1a3d08922c535239ca1913859b2e713ee2437327fa6be59b022
Sep 09 13:42:17.126552 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:17.126536    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:42:17.892721 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:17.892688    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" event={"ID":"5ed68bc7-aa6b-4710-b598-295470b764a7","Type":"ContainerStarted","Data":"622bfc6e98cdb1a3d08922c535239ca1913859b2e713ee2437327fa6be59b022"}
Sep 09 13:42:18.900500 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:18.900459    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" event={"ID":"5ed68bc7-aa6b-4710-b598-295470b764a7","Type":"ContainerStarted","Data":"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"}
Sep 09 13:42:18.900977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:18.900606    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:42:18.921018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:18.920959    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" podStartSLOduration=2.788852308 podStartE2EDuration="3.920941674s" totalImagesPullingTime="1.132089366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:42:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:42:18.917953915 +0000 UTC m=+957.806653905" watchObservedRunningTime="2026-09-09 13:42:18.920941674 +0000 UTC m=+957.809641663"
Sep 09 13:42:24.911611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:42:24.911560    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:43:23.165465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.165043    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:43:23.169987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.169962    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.172886 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.172682    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-905b058d-serving-cert\""
Sep 09 13:43:23.181590 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.181555    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:43:23.348270 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.348232    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.348450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.348278    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxp5q\" (UniqueName: \"kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.348450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.348362    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.348450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.348409    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.449191 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.449122    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.449191 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.449156    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.449398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.449212    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.449398 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.449232    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxp5q\" (UniqueName: \"kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.449736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.449713    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.451529 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.451505    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.451628 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.451561    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.456842 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.456821    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxp5q\" (UniqueName: \"kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q\") pod \"maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.482852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.482828    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:23.765650 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:23.765625    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:43:23.766984 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:43:23.766955    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372dc21d_c6b8_447c_b794_c71d5593dc90.slice/crio-b137e1d28ea0492ec080368a56ad95e97ad967167005d82425f5157a5c9084ed WatchSource:0}: Error finding container b137e1d28ea0492ec080368a56ad95e97ad967167005d82425f5157a5c9084ed: Status 404 returned error can't find the container with id b137e1d28ea0492ec080368a56ad95e97ad967167005d82425f5157a5c9084ed
Sep 09 13:43:24.324798 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:24.324767    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" event={"ID":"372dc21d-c6b8-447c-b794-c71d5593dc90","Type":"ContainerStarted","Data":"b137e1d28ea0492ec080368a56ad95e97ad967167005d82425f5157a5c9084ed"}
Sep 09 13:43:25.332533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:25.332496    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" event={"ID":"372dc21d-c6b8-447c-b794-c71d5593dc90","Type":"ContainerStarted","Data":"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"}
Sep 09 13:43:25.333024 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:25.332556    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:25.352991 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:25.352945    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" podStartSLOduration=1.8368535860000001 podStartE2EDuration="2.352930153s" totalImagesPullingTime="516.076567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:43:25.351547032 +0000 UTC m=+1024.240247050" watchObservedRunningTime="2026-09-09 13:43:25.352930153 +0000 UTC m=+1024.241630143"
Sep 09 13:43:31.343987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:31.343951    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:43:41.691826 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:41.691790    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:43:41.692287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:41.692040    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" podUID="148de768-f804-42d1-bd6a-4e1a1f719731" containerName="maas-api" containerID="cri-o://164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad" gracePeriod=30
Sep 09 13:43:41.784518 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:41.784483    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:43:41.784772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:41.784744    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" podUID="8ccdd0a3-0844-4f93-8721-adc220f3238c" containerName="payload-pre-processing" containerID="cri-o://e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa" gracePeriod=30
Sep 09 13:43:42.054181 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.054153    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:43:42.116238 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.116190    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume\") pod \"8ccdd0a3-0844-4f93-8721-adc220f3238c\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") "
Sep 09 13:43:42.116437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.116276    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzd56\" (UniqueName: \"kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56\") pod \"8ccdd0a3-0844-4f93-8721-adc220f3238c\" (UID: \"8ccdd0a3-0844-4f93-8721-adc220f3238c\") "
Sep 09 13:43:42.116718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.116681    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8ccdd0a3-0844-4f93-8721-adc220f3238c" (UID: "8ccdd0a3-0844-4f93-8721-adc220f3238c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:43:42.119472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.119434    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56" (OuterVolumeSpecName: "kube-api-access-hzd56") pod "8ccdd0a3-0844-4f93-8721-adc220f3238c" (UID: "8ccdd0a3-0844-4f93-8721-adc220f3238c"). InnerVolumeSpecName "kube-api-access-hzd56". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:43:42.153686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.153663    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:43:42.217180 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217142    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7z5t\" (UniqueName: \"kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t\") pod \"148de768-f804-42d1-bd6a-4e1a1f719731\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") "
Sep 09 13:43:42.217364 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217224    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs\") pod \"148de768-f804-42d1-bd6a-4e1a1f719731\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") "
Sep 09 13:43:42.217364 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217265    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls\") pod \"148de768-f804-42d1-bd6a-4e1a1f719731\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") "
Sep 09 13:43:42.217364 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217331    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca\") pod \"148de768-f804-42d1-bd6a-4e1a1f719731\" (UID: \"148de768-f804-42d1-bd6a-4e1a1f719731\") "
Sep 09 13:43:42.217622 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217566    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8ccdd0a3-0844-4f93-8721-adc220f3238c-plugins-config-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.217701 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217629    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzd56\" (UniqueName: \"kubernetes.io/projected/8ccdd0a3-0844-4f93-8721-adc220f3238c-kube-api-access-hzd56\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.217827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.217794    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "148de768-f804-42d1-bd6a-4e1a1f719731" (UID: "148de768-f804-42d1-bd6a-4e1a1f719731"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:43:42.219520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.219490    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "148de768-f804-42d1-bd6a-4e1a1f719731" (UID: "148de768-f804-42d1-bd6a-4e1a1f719731"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:43:42.219809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.219784    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "148de768-f804-42d1-bd6a-4e1a1f719731" (UID: "148de768-f804-42d1-bd6a-4e1a1f719731"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:43:42.219809 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.219802    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t" (OuterVolumeSpecName: "kube-api-access-m7z5t") pod "148de768-f804-42d1-bd6a-4e1a1f719731" (UID: "148de768-f804-42d1-bd6a-4e1a1f719731"). InnerVolumeSpecName "kube-api-access-m7z5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:43:42.318446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.318417    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.318446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.318444    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/148de768-f804-42d1-bd6a-4e1a1f719731-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.318446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.318455    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7z5t\" (UniqueName: \"kubernetes.io/projected/148de768-f804-42d1-bd6a-4e1a1f719731-kube-api-access-m7z5t\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.318670 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.318463    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/148de768-f804-42d1-bd6a-4e1a1f719731-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:42.437125 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.437089    2559 generic.go:356] "Generic (PLEG): container finished" podID="148de768-f804-42d1-bd6a-4e1a1f719731" containerID="164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad" exitCode=0
Sep 09 13:43:42.437296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.437140    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"
Sep 09 13:43:42.437296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.437174    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" event={"ID":"148de768-f804-42d1-bd6a-4e1a1f719731","Type":"ContainerDied","Data":"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"}
Sep 09 13:43:42.437296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.437252    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv" event={"ID":"148de768-f804-42d1-bd6a-4e1a1f719731","Type":"ContainerDied","Data":"e1d34073258f21843d946e13233d022779c62b94af3e30666895a5e7ca586f5e"}
Sep 09 13:43:42.437296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.437277    2559 scope.go:117] "RemoveContainer" containerID="164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"
Sep 09 13:43:42.438771 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.438740    2559 generic.go:356] "Generic (PLEG): container finished" podID="8ccdd0a3-0844-4f93-8721-adc220f3238c" containerID="e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa" exitCode=0
Sep 09 13:43:42.438920 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.438783    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"
Sep 09 13:43:42.438920 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.438776    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" event={"ID":"8ccdd0a3-0844-4f93-8721-adc220f3238c","Type":"ContainerDied","Data":"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"}
Sep 09 13:43:42.439054 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.438956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc" event={"ID":"8ccdd0a3-0844-4f93-8721-adc220f3238c","Type":"ContainerDied","Data":"5450777a2af1795850be3dfd234d4ca2ad49c6e737cc396a613e0d44884d4a4d"}
Sep 09 13:43:42.447931 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.447912    2559 scope.go:117] "RemoveContainer" containerID="164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"
Sep 09 13:43:42.448227 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:43:42.448203    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad\": container with ID starting with 164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad not found: ID does not exist" containerID="164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"
Sep 09 13:43:42.448325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.448235    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad"} err="failed to get container status \"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad\": rpc error: code = NotFound desc = could not find container \"164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad\": container with ID starting with 164774b318260b2d065dcc15a48a0af7dab162280c1a88a8b8895e3c892d46ad not found: ID does not exist"
Sep 09 13:43:42.448325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.448255    2559 scope.go:117] "RemoveContainer" containerID="e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"
Sep 09 13:43:42.456469 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.456453    2559 scope.go:117] "RemoveContainer" containerID="e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"
Sep 09 13:43:42.456749 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:43:42.456728    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa\": container with ID starting with e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa not found: ID does not exist" containerID="e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"
Sep 09 13:43:42.456823 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.456753    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa"} err="failed to get container status \"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa\": rpc error: code = NotFound desc = could not find container \"e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa\": container with ID starting with e0d6105ebe0d4d15bef97bc234b71ee24b629889fdd95b15952c85c21e8be5fa not found: ID does not exist"
Sep 09 13:43:42.465000 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.464979    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:43:42.470336 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.470288    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4f59a0-798764d6cb-8c7sv"]
Sep 09 13:43:42.481047 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.481029    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:43:42.490465 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.490445    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-4f59a0-6dc6d8758-8g6cc"]
Sep 09 13:43:42.958044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.957977    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:43:42.958461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:42.958233    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" podUID="5ed68bc7-aa6b-4710-b598-295470b764a7" containerName="maas-api" containerID="cri-o://0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd" gracePeriod=30
Sep 09 13:43:43.218496 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.218458    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:43:43.327347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.327273    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs\") pod \"5ed68bc7-aa6b-4710-b598-295470b764a7\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") "
Sep 09 13:43:43.327347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.327327    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca\") pod \"5ed68bc7-aa6b-4710-b598-295470b764a7\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") "
Sep 09 13:43:43.327547 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.327407    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") pod \"5ed68bc7-aa6b-4710-b598-295470b764a7\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") "
Sep 09 13:43:43.327547 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.327438    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v5dx\" (UniqueName: \"kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx\") pod \"5ed68bc7-aa6b-4710-b598-295470b764a7\" (UID: \"5ed68bc7-aa6b-4710-b598-295470b764a7\") "
Sep 09 13:43:43.327773 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.327742    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5ed68bc7-aa6b-4710-b598-295470b764a7" (UID: "5ed68bc7-aa6b-4710-b598-295470b764a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:43:43.329732 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.329695    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5ed68bc7-aa6b-4710-b598-295470b764a7" (UID: "5ed68bc7-aa6b-4710-b598-295470b764a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:43:43.330013 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.329990    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5ed68bc7-aa6b-4710-b598-295470b764a7" (UID: "5ed68bc7-aa6b-4710-b598-295470b764a7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:43:43.330128 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.330017    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx" (OuterVolumeSpecName: "kube-api-access-8v5dx") pod "5ed68bc7-aa6b-4710-b598-295470b764a7" (UID: "5ed68bc7-aa6b-4710-b598-295470b764a7"). InnerVolumeSpecName "kube-api-access-8v5dx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:43:43.429323 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.429289    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:43.429323 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.429315    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ed68bc7-aa6b-4710-b598-295470b764a7-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:43.429323 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.429327    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5ed68bc7-aa6b-4710-b598-295470b764a7-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:43.429550 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.429337    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v5dx\" (UniqueName: \"kubernetes.io/projected/5ed68bc7-aa6b-4710-b598-295470b764a7-kube-api-access-8v5dx\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:43:43.448524 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.448493    2559 generic.go:356] "Generic (PLEG): container finished" podID="5ed68bc7-aa6b-4710-b598-295470b764a7" containerID="0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd" exitCode=0
Sep 09 13:43:43.448686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.448538    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"
Sep 09 13:43:43.448686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.448595    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" event={"ID":"5ed68bc7-aa6b-4710-b598-295470b764a7","Type":"ContainerDied","Data":"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"}
Sep 09 13:43:43.448686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.448643    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s" event={"ID":"5ed68bc7-aa6b-4710-b598-295470b764a7","Type":"ContainerDied","Data":"622bfc6e98cdb1a3d08922c535239ca1913859b2e713ee2437327fa6be59b022"}
Sep 09 13:43:43.448686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.448667    2559 scope.go:117] "RemoveContainer" containerID="0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"
Sep 09 13:43:43.460629 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.460610    2559 scope.go:117] "RemoveContainer" containerID="0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"
Sep 09 13:43:43.460893 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:43:43.460874    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd\": container with ID starting with 0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd not found: ID does not exist" containerID="0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"
Sep 09 13:43:43.460949 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.460901    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd"} err="failed to get container status \"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd\": rpc error: code = NotFound desc = could not find container \"0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd\": container with ID starting with 0f356ee584a06776d838fc5baa9a2f6262d617ed63266dd2f5acb190912602cd not found: ID does not exist"
Sep 09 13:43:43.474511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.474485    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:43:43.479734 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.479696    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-c5f375-7bdcc9b69c-cqq5s"]
Sep 09 13:43:43.618932 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.618861    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="148de768-f804-42d1-bd6a-4e1a1f719731" path="/var/lib/kubelet/pods/148de768-f804-42d1-bd6a-4e1a1f719731/volumes"
Sep 09 13:43:43.619273 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.619257    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ed68bc7-aa6b-4710-b598-295470b764a7" path="/var/lib/kubelet/pods/5ed68bc7-aa6b-4710-b598-295470b764a7/volumes"
Sep 09 13:43:43.619614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:43:43.619602    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ccdd0a3-0844-4f93-8721-adc220f3238c" path="/var/lib/kubelet/pods/8ccdd0a3-0844-4f93-8721-adc220f3238c/volumes"
Sep 09 13:44:00.399445 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.399405    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:44:00.399948 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.399702    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" podUID="7b9befd5-55b6-40c6-94b3-03564d6467ae" containerName="payload-pre-processing" containerID="cri-o://8b8666a758267cf04b33ecc540c9cdd87a9549d0bf5a236a5cc4343e3a4e045c" gracePeriod=30
Sep 09 13:44:00.556237 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.556204    2559 generic.go:356] "Generic (PLEG): container finished" podID="7b9befd5-55b6-40c6-94b3-03564d6467ae" containerID="8b8666a758267cf04b33ecc540c9cdd87a9549d0bf5a236a5cc4343e3a4e045c" exitCode=0
Sep 09 13:44:00.556403 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.556276    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" event={"ID":"7b9befd5-55b6-40c6-94b3-03564d6467ae","Type":"ContainerDied","Data":"8b8666a758267cf04b33ecc540c9cdd87a9549d0bf5a236a5cc4343e3a4e045c"}
Sep 09 13:44:00.666785 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.666729    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:44:00.784696 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.784668    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq\") pod \"7b9befd5-55b6-40c6-94b3-03564d6467ae\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") "
Sep 09 13:44:00.784850 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.784763    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume\") pod \"7b9befd5-55b6-40c6-94b3-03564d6467ae\" (UID: \"7b9befd5-55b6-40c6-94b3-03564d6467ae\") "
Sep 09 13:44:00.785132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.785109    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7b9befd5-55b6-40c6-94b3-03564d6467ae" (UID: "7b9befd5-55b6-40c6-94b3-03564d6467ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:44:00.786742 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.786714    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq" (OuterVolumeSpecName: "kube-api-access-s5sxq") pod "7b9befd5-55b6-40c6-94b3-03564d6467ae" (UID: "7b9befd5-55b6-40c6-94b3-03564d6467ae"). InnerVolumeSpecName "kube-api-access-s5sxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:44:00.886094 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.886069    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5sxq\" (UniqueName: \"kubernetes.io/projected/7b9befd5-55b6-40c6-94b3-03564d6467ae-kube-api-access-s5sxq\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:00.886094 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:00.886094    2559 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7b9befd5-55b6-40c6-94b3-03564d6467ae-plugins-config-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:01.563556 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.563524    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t" event={"ID":"7b9befd5-55b6-40c6-94b3-03564d6467ae","Type":"ContainerDied","Data":"b68b66bb54da675424e508cd21ba40f8cc68bc2cc2e370f4ebd141879a1a9b1e"}
Sep 09 13:44:01.564036 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.563560    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"
Sep 09 13:44:01.564036 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.563651    2559 scope.go:117] "RemoveContainer" containerID="8b8666a758267cf04b33ecc540c9cdd87a9549d0bf5a236a5cc4343e3a4e045c"
Sep 09 13:44:01.586657 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.586635    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:44:01.594706 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.594682    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-093e38-7f5d966c54-jxf7t"]
Sep 09 13:44:01.617844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:01.617806    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b9befd5-55b6-40c6-94b3-03564d6467ae" path="/var/lib/kubelet/pods/7b9befd5-55b6-40c6-94b3-03564d6467ae/volumes"
Sep 09 13:44:05.129679 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.129639    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:44:05.130144 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.129869    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" podUID="372dc21d-c6b8-447c-b794-c71d5593dc90" containerName="maas-api" containerID="cri-o://072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8" gracePeriod=30
Sep 09 13:44:05.380349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.380295    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:44:05.529597 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.529539    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxp5q\" (UniqueName: \"kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q\") pod \"372dc21d-c6b8-447c-b794-c71d5593dc90\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") "
Sep 09 13:44:05.529786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.529631    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs\") pod \"372dc21d-c6b8-447c-b794-c71d5593dc90\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") "
Sep 09 13:44:05.529786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.529660    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls\") pod \"372dc21d-c6b8-447c-b794-c71d5593dc90\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") "
Sep 09 13:44:05.529786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.529695    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca\") pod \"372dc21d-c6b8-447c-b794-c71d5593dc90\" (UID: \"372dc21d-c6b8-447c-b794-c71d5593dc90\") "
Sep 09 13:44:05.530360 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.530299    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "372dc21d-c6b8-447c-b794-c71d5593dc90" (UID: "372dc21d-c6b8-447c-b794-c71d5593dc90"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:44:05.531957 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.531922    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "372dc21d-c6b8-447c-b794-c71d5593dc90" (UID: "372dc21d-c6b8-447c-b794-c71d5593dc90"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:44:05.531957 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.531938    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "372dc21d-c6b8-447c-b794-c71d5593dc90" (UID: "372dc21d-c6b8-447c-b794-c71d5593dc90"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:44:05.532137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.531971    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q" (OuterVolumeSpecName: "kube-api-access-lxp5q") pod "372dc21d-c6b8-447c-b794-c71d5593dc90" (UID: "372dc21d-c6b8-447c-b794-c71d5593dc90"). InnerVolumeSpecName "kube-api-access-lxp5q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:44:05.589358 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.589327    2559 generic.go:356] "Generic (PLEG): container finished" podID="372dc21d-c6b8-447c-b794-c71d5593dc90" containerID="072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8" exitCode=0
Sep 09 13:44:05.589479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.589374    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"
Sep 09 13:44:05.589479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.589406    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" event={"ID":"372dc21d-c6b8-447c-b794-c71d5593dc90","Type":"ContainerDied","Data":"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"}
Sep 09 13:44:05.589479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.589457    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6" event={"ID":"372dc21d-c6b8-447c-b794-c71d5593dc90","Type":"ContainerDied","Data":"b137e1d28ea0492ec080368a56ad95e97ad967167005d82425f5157a5c9084ed"}
Sep 09 13:44:05.589627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.589486    2559 scope.go:117] "RemoveContainer" containerID="072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"
Sep 09 13:44:05.598818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.598799    2559 scope.go:117] "RemoveContainer" containerID="072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"
Sep 09 13:44:05.599095 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:44:05.599074    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8\": container with ID starting with 072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8 not found: ID does not exist" containerID="072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"
Sep 09 13:44:05.599148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.599104    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8"} err="failed to get container status \"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8\": rpc error: code = NotFound desc = could not find container \"072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8\": container with ID starting with 072edfebc1255d407a4fa851ec3bb8b8506c88d7c229ac7eb18242f5c2ababd8 not found: ID does not exist"
Sep 09 13:44:05.618968 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.618944    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:44:05.619250 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.619229    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-905b058d-79d5b9c9d-wj2q6"]
Sep 09 13:44:05.631436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.631385    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxp5q\" (UniqueName: \"kubernetes.io/projected/372dc21d-c6b8-447c-b794-c71d5593dc90-kube-api-access-lxp5q\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:05.631436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.631407    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:05.631436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.631418    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/372dc21d-c6b8-447c-b794-c71d5593dc90-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:05.631436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:05.631431    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/372dc21d-c6b8-447c-b794-c71d5593dc90-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:44:07.616893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:07.616846    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="372dc21d-c6b8-447c-b794-c71d5593dc90" path="/var/lib/kubelet/pods/372dc21d-c6b8-447c-b794-c71d5593dc90/volumes"
Sep 09 13:44:30.682394 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.682363    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:44:30.683476 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683450    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ed68bc7-aa6b-4710-b598-295470b764a7" containerName="maas-api"
Sep 09 13:44:30.683476 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683474    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ed68bc7-aa6b-4710-b598-295470b764a7" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683483    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="372dc21d-c6b8-447c-b794-c71d5593dc90" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683489    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="372dc21d-c6b8-447c-b794-c71d5593dc90" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683506    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9befd5-55b6-40c6-94b3-03564d6467ae" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683511    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9befd5-55b6-40c6-94b3-03564d6467ae" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683524    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="148de768-f804-42d1-bd6a-4e1a1f719731" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683530    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="148de768-f804-42d1-bd6a-4e1a1f719731" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683561    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ccdd0a3-0844-4f93-8721-adc220f3238c" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683586    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ccdd0a3-0844-4f93-8721-adc220f3238c" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683687    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ed68bc7-aa6b-4710-b598-295470b764a7" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683699    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="372dc21d-c6b8-447c-b794-c71d5593dc90" containerName="maas-api"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683717    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ccdd0a3-0844-4f93-8721-adc220f3238c" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683722    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9befd5-55b6-40c6-94b3-03564d6467ae" containerName="payload-pre-processing"
Sep 09 13:44:30.683733 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.683727    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="148de768-f804-42d1-bd6a-4e1a1f719731" containerName="maas-api"
Sep 09 13:44:30.687088 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.687065    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.689791 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.689771    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b5cba4-serving-cert\""
Sep 09 13:44:30.698924 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.697558    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:44:30.768198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.768166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.768362 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.768214    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhnw\" (UniqueName: \"kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.768412 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.768361    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.768412 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.768403    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.869019 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.868985    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.869195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.869029    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.869195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.869058    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.869195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.869076    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhnw\" (UniqueName: \"kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.869680 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.869650    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.871486 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.871463    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.871651 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.871630    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:30.880511 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:30.880491    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhnw\" (UniqueName: \"kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw\") pod \"maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:31.002723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:31.002696    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:31.282841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:31.282810    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:44:31.284298 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:44:31.284261    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cf250f_b749_48df_a7e5_f6a42cbb1774.slice/crio-5cea9578e8628af4b90012ebc3b7d0426eadf28e79914a39c6ff4ee2cfb28d96 WatchSource:0}: Error finding container 5cea9578e8628af4b90012ebc3b7d0426eadf28e79914a39c6ff4ee2cfb28d96: Status 404 returned error can't find the container with id 5cea9578e8628af4b90012ebc3b7d0426eadf28e79914a39c6ff4ee2cfb28d96
Sep 09 13:44:31.748001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:31.747963    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" event={"ID":"b5cf250f-b749-48df-a7e5-f6a42cbb1774","Type":"ContainerStarted","Data":"5cea9578e8628af4b90012ebc3b7d0426eadf28e79914a39c6ff4ee2cfb28d96"}
Sep 09 13:44:32.757204 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:32.757170    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" event={"ID":"b5cf250f-b749-48df-a7e5-f6a42cbb1774","Type":"ContainerStarted","Data":"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"}
Sep 09 13:44:32.757560 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:32.757315    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:32.775207 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:32.775140    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" podStartSLOduration=1.7712971990000002 podStartE2EDuration="2.775119231s" totalImagesPullingTime="1.003822032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:44:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:44:32.774671697 +0000 UTC m=+1091.663371689" watchObservedRunningTime="2026-09-09 13:44:32.775119231 +0000 UTC m=+1091.663819221"
Sep 09 13:44:38.768343 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:38.768317    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:44:41.576909 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.576870    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:44:41.580883 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.580854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.583848 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.583822    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-df48ae-serving-cert\""
Sep 09 13:44:41.594285 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.594259    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:44:41.678729 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.678699    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.678871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.678732    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.678871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.678766    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.678947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.678891    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm98t\" (UniqueName: \"kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.779716 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.779687    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.779716 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.779718    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.779907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.779753    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.779907 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.779805    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm98t\" (UniqueName: \"kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.779993 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:44:41.779917    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-serving-cert: secret "maas-api-e2e-shared-b-w3-df48ae-serving-cert" not found
Sep 09 13:44:41.779993 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:44:41.779990    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls podName:99cbbb71-5088-479b-856c-536f7af2bd9f nodeName:}" failed. No retries permitted until 2026-09-09 13:44:42.279966095 +0000 UTC m=+1101.168666078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls") pod "maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" (UID: "99cbbb71-5088-479b-856c-536f7af2bd9f") : secret "maas-api-e2e-shared-b-w3-df48ae-serving-cert" not found
Sep 09 13:44:41.780494 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.780458    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.782281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.782259    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:41.788669 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:41.788647    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm98t\" (UniqueName: \"kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:42.284498 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:42.284461    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:42.286852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:42.286825    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:42.495210 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:42.495174    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:42.768706 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:42.768674    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:44:42.769749 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:44:42.769720    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cbbb71_5088_479b_856c_536f7af2bd9f.slice/crio-7633418c434fc7bf545bca62cea6c9ecb404a45b7352b548b77db2a6fb2b8e19 WatchSource:0}: Error finding container 7633418c434fc7bf545bca62cea6c9ecb404a45b7352b548b77db2a6fb2b8e19: Status 404 returned error can't find the container with id 7633418c434fc7bf545bca62cea6c9ecb404a45b7352b548b77db2a6fb2b8e19
Sep 09 13:44:42.817375 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:42.817297    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" event={"ID":"99cbbb71-5088-479b-856c-536f7af2bd9f","Type":"ContainerStarted","Data":"7633418c434fc7bf545bca62cea6c9ecb404a45b7352b548b77db2a6fb2b8e19"}
Sep 09 13:44:43.824618 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:43.824567    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" event={"ID":"99cbbb71-5088-479b-856c-536f7af2bd9f","Type":"ContainerStarted","Data":"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"}
Sep 09 13:44:43.825001 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:43.824646    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:43.844534 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:43.844486    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" podStartSLOduration=2.341436944 podStartE2EDuration="2.844471333s" totalImagesPullingTime="503.034389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:44:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:44:43.843348642 +0000 UTC m=+1102.732048632" watchObservedRunningTime="2026-09-09 13:44:43.844471333 +0000 UTC m=+1102.733171322"
Sep 09 13:44:49.837201 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:49.837171    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:44:58.731563 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.731523    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:44:58.735290 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.735259    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.740314 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.740288    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-6a7dbc-serving-cert\""
Sep 09 13:44:58.755658 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.755628    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:44:58.838021 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.837990    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.838208 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.838045    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.838208 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.838120    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4lkj\" (UniqueName: \"kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.838208 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.838165    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.938815 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.938734    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.938815 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.938784    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.939000 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:44:58.938878    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-serving-cert: secret "maas-api-e2e-shared-b-w2-6a7dbc-serving-cert" not found
Sep 09 13:44:58.939000 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.938912    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4lkj\" (UniqueName: \"kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.939000 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:44:58.938939    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls podName:e13b2ea3-eb7c-431c-9d17-7f315f25dd33 nodeName:}" failed. No retries permitted until 2026-09-09 13:44:59.43892081 +0000 UTC m=+1118.327620797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls") pod "maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" (UID: "e13b2ea3-eb7c-431c-9d17-7f315f25dd33") : secret "maas-api-e2e-shared-b-w2-6a7dbc-serving-cert" not found
Sep 09 13:44:58.939000 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.938959    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.939473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.939443    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.941454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.941433    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:58.950790 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:58.950768    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4lkj\" (UniqueName: \"kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:59.444893 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:59.444857    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:59.447747 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:59.447719    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:59.649301 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:59.649264    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:44:59.925503 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:44:59.925475    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:44:59.925842 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:44:59.925821    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13b2ea3_eb7c_431c_9d17_7f315f25dd33.slice/crio-b7d879e1f152e6e60f26407d3abcbbd0d9710b00d5eaa46b3cb20138758a64c1 WatchSource:0}: Error finding container b7d879e1f152e6e60f26407d3abcbbd0d9710b00d5eaa46b3cb20138758a64c1: Status 404 returned error can't find the container with id b7d879e1f152e6e60f26407d3abcbbd0d9710b00d5eaa46b3cb20138758a64c1
Sep 09 13:45:00.926512 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:00.926473    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" event={"ID":"e13b2ea3-eb7c-431c-9d17-7f315f25dd33","Type":"ContainerStarted","Data":"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"}
Sep 09 13:45:00.926938 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:00.926527    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" event={"ID":"e13b2ea3-eb7c-431c-9d17-7f315f25dd33","Type":"ContainerStarted","Data":"b7d879e1f152e6e60f26407d3abcbbd0d9710b00d5eaa46b3cb20138758a64c1"}
Sep 09 13:45:00.926938 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:00.926722    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:45:00.947679 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:00.947634    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" podStartSLOduration=2.579475226 podStartE2EDuration="2.947620605s" totalImagesPullingTime="368.145379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:44:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:00.94478976 +0000 UTC m=+1119.833489762" watchObservedRunningTime="2026-09-09 13:45:00.947620605 +0000 UTC m=+1119.836320594"
Sep 09 13:45:06.938118 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:06.938085    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:45:23.351709 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.351676    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7937-4e1d8b"]
Sep 09 13:45:23.355373 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.355354    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:23.358771 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.358627    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qg8jq\""
Sep 09 13:45:23.362108 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.362061    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-4e1d8b"]
Sep 09 13:45:23.477992 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.477962    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6gp\" (UniqueName: \"kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp\") pod \"test-curl-7937-4e1d8b\" (UID: \"87646093-daaf-4f94-9518-01e0dc4ebe82\") " pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:23.578748 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.578717    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6gp\" (UniqueName: \"kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp\") pod \"test-curl-7937-4e1d8b\" (UID: \"87646093-daaf-4f94-9518-01e0dc4ebe82\") " pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:23.588179 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.588150    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6gp\" (UniqueName: \"kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp\") pod \"test-curl-7937-4e1d8b\" (UID: \"87646093-daaf-4f94-9518-01e0dc4ebe82\") " pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:23.667436 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.667377    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:23.938474 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:23.938447    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-4e1d8b"]
Sep 09 13:45:23.939244 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:45:23.939206    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87646093_daaf_4f94_9518_01e0dc4ebe82.slice/crio-1dc297da66b500a556595c3d5608775f820764b96a83d5587d7da99266f4014f WatchSource:0}: Error finding container 1dc297da66b500a556595c3d5608775f820764b96a83d5587d7da99266f4014f: Status 404 returned error can't find the container with id 1dc297da66b500a556595c3d5608775f820764b96a83d5587d7da99266f4014f
Sep 09 13:45:24.071305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.071268    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-4e1d8b" event={"ID":"87646093-daaf-4f94-9518-01e0dc4ebe82","Type":"ContainerStarted","Data":"d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507"}
Sep 09 13:45:24.071399 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.071319    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-4e1d8b" event={"ID":"87646093-daaf-4f94-9518-01e0dc4ebe82","Type":"ContainerStarted","Data":"1dc297da66b500a556595c3d5608775f820764b96a83d5587d7da99266f4014f"}
Sep 09 13:45:24.089556 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.089515    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7937-4e1d8b" podStartSLOduration=1.089499553 podStartE2EDuration="1.089499553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:45:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:24.086013707 +0000 UTC m=+1142.974713697" watchObservedRunningTime="2026-09-09 13:45:24.089499553 +0000 UTC m=+1142.978199543"
Sep 09 13:45:24.699972 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.699896    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7937-4e1d8b"]
Sep 09 13:45:24.704606 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.704560    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7937-4e1d8b"]
Sep 09 13:45:24.824802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.824770    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7937-cc02cb"]
Sep 09 13:45:24.825223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.825209    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" containerName="test-curl-7937-4e1d8b"
Sep 09 13:45:24.825270 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.825225    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" containerName="test-curl-7937-4e1d8b"
Sep 09 13:45:24.825306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.825302    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" containerName="test-curl-7937-4e1d8b"
Sep 09 13:45:24.828505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.828488    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:24.835655 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.835546    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-cc02cb"]
Sep 09 13:45:24.993795 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:24.993759    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdsbk\" (UniqueName: \"kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk\") pod \"test-curl-7937-cc02cb\" (UID: \"0887a094-3f9e-4bae-87c6-26aac1bf1773\") " pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:25.077503 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:25.077476    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7937-4e1d8b" container="test-curl-7937-4e1d8b"
Sep 09 13:45:25.095181 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:25.095140    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdsbk\" (UniqueName: \"kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk\") pod \"test-curl-7937-cc02cb\" (UID: \"0887a094-3f9e-4bae-87c6-26aac1bf1773\") " pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:25.104410 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:25.104380    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdsbk\" (UniqueName: \"kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk\") pod \"test-curl-7937-cc02cb\" (UID: \"0887a094-3f9e-4bae-87c6-26aac1bf1773\") " pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:25.139670 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:25.139644    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:25.410382 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:25.410350    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-cc02cb"]
Sep 09 13:45:25.411349 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:45:25.411319    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0887a094_3f9e_4bae_87c6_26aac1bf1773.slice/crio-1c62254e8c34c951e4c2a5de289bf24efb4c23350072122533667529f7d69b12 WatchSource:0}: Error finding container 1c62254e8c34c951e4c2a5de289bf24efb4c23350072122533667529f7d69b12: Status 404 returned error can't find the container with id 1c62254e8c34c951e4c2a5de289bf24efb4c23350072122533667529f7d69b12
Sep 09 13:45:26.085320 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.085278    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-cc02cb" event={"ID":"0887a094-3f9e-4bae-87c6-26aac1bf1773","Type":"ContainerStarted","Data":"007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f"}
Sep 09 13:45:26.085777 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.085330    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-cc02cb" event={"ID":"0887a094-3f9e-4bae-87c6-26aac1bf1773","Type":"ContainerStarted","Data":"1c62254e8c34c951e4c2a5de289bf24efb4c23350072122533667529f7d69b12"}
Sep 09 13:45:26.085777 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.085383    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7937-4e1d8b" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" containerName="test-curl-7937-4e1d8b" containerID="cri-o://d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507" gracePeriod=2
Sep 09 13:45:26.103033 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.102982    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7937-cc02cb" podStartSLOduration=2.102970211 podStartE2EDuration="2.102970211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:45:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:26.101300489 +0000 UTC m=+1144.990000478" watchObservedRunningTime="2026-09-09 13:45:26.102970211 +0000 UTC m=+1144.991670201"
Sep 09 13:45:26.715633 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.715552    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7937-cc02cb"]
Sep 09 13:45:26.720000 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.719971    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7937-cc02cb"]
Sep 09 13:45:26.831017 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.830945    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7937-eaf3c1"]
Sep 09 13:45:26.831423 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.831409    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" containerName="test-curl-7937-cc02cb"
Sep 09 13:45:26.831468 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.831425    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" containerName="test-curl-7937-cc02cb"
Sep 09 13:45:26.831503 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.831499    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" containerName="test-curl-7937-cc02cb"
Sep 09 13:45:26.834650 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.834635    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:26.839192 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.839168    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-eaf3c1"]
Sep 09 13:45:26.909886 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:26.909859    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcbqm\" (UniqueName: \"kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm\") pod \"test-curl-7937-eaf3c1\" (UID: \"4167e1c6-feb3-492d-9da7-bd42ab504399\") " pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:27.010664 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:27.010641    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcbqm\" (UniqueName: \"kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm\") pod \"test-curl-7937-eaf3c1\" (UID: \"4167e1c6-feb3-492d-9da7-bd42ab504399\") " pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:27.019495 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:27.019471    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcbqm\" (UniqueName: \"kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm\") pod \"test-curl-7937-eaf3c1\" (UID: \"4167e1c6-feb3-492d-9da7-bd42ab504399\") " pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:27.090925 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:27.090873    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7937-cc02cb" container="test-curl-7937-cc02cb"
Sep 09 13:45:27.146370 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:27.146348    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:27.419471 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:27.419439    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-eaf3c1"]
Sep 09 13:45:27.419684 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:45:27.419655    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4167e1c6_feb3_492d_9da7_bd42ab504399.slice/crio-5342c938de6dc60e6efe2ae8a7f3042eea4da2fd49fb54f75a6c808f3405604c WatchSource:0}: Error finding container 5342c938de6dc60e6efe2ae8a7f3042eea4da2fd49fb54f75a6c808f3405604c: Status 404 returned error can't find the container with id 5342c938de6dc60e6efe2ae8a7f3042eea4da2fd49fb54f75a6c808f3405604c
Sep 09 13:45:28.101851 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.101817    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-eaf3c1" event={"ID":"4167e1c6-feb3-492d-9da7-bd42ab504399","Type":"ContainerStarted","Data":"0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a"}
Sep 09 13:45:28.102213 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.101867    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-eaf3c1" event={"ID":"4167e1c6-feb3-492d-9da7-bd42ab504399","Type":"ContainerStarted","Data":"5342c938de6dc60e6efe2ae8a7f3042eea4da2fd49fb54f75a6c808f3405604c"}
Sep 09 13:45:28.102213 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.101942    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7937-cc02cb" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" containerName="test-curl-7937-cc02cb" containerID="cri-o://007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f" gracePeriod=2
Sep 09 13:45:28.119036 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.118986    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7937-eaf3c1" podStartSLOduration=2.118971624 podStartE2EDuration="2.118971624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:45:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:28.116613719 +0000 UTC m=+1147.005313706" watchObservedRunningTime="2026-09-09 13:45:28.118971624 +0000 UTC m=+1147.007671614"
Sep 09 13:45:28.259820 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.259792    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:28.424397 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.424323    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x6gp\" (UniqueName: \"kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp\") pod \"87646093-daaf-4f94-9518-01e0dc4ebe82\" (UID: \"87646093-daaf-4f94-9518-01e0dc4ebe82\") "
Sep 09 13:45:28.426204 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.426176    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp" (OuterVolumeSpecName: "kube-api-access-4x6gp") pod "87646093-daaf-4f94-9518-01e0dc4ebe82" (UID: "87646093-daaf-4f94-9518-01e0dc4ebe82"). InnerVolumeSpecName "kube-api-access-4x6gp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:45:28.525380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.525355    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x6gp\" (UniqueName: \"kubernetes.io/projected/87646093-daaf-4f94-9518-01e0dc4ebe82-kube-api-access-4x6gp\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:45:28.718317 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.718280    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7937-eaf3c1"]
Sep 09 13:45:28.722515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.722483    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7937-eaf3c1"]
Sep 09 13:45:28.842567 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.842527    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7937-948786"]
Sep 09 13:45:28.843093 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.843071    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" containerName="test-curl-7937-eaf3c1"
Sep 09 13:45:28.843195 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.843095    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" containerName="test-curl-7937-eaf3c1"
Sep 09 13:45:28.843263 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.843212    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" containerName="test-curl-7937-eaf3c1"
Sep 09 13:45:28.846281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.846261    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:28.853222 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.853188    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-948786"]
Sep 09 13:45:28.928929 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:28.928900    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrcfx\" (UniqueName: \"kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx\") pod \"test-curl-7937-948786\" (UID: \"2687c673-89bf-4b1d-bf0d-6980f7e65485\") " pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:29.030141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.030052    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrcfx\" (UniqueName: \"kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx\") pod \"test-curl-7937-948786\" (UID: \"2687c673-89bf-4b1d-bf0d-6980f7e65485\") " pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:29.038997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.038969    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrcfx\" (UniqueName: \"kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx\") pod \"test-curl-7937-948786\" (UID: \"2687c673-89bf-4b1d-bf0d-6980f7e65485\") " pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:29.109440 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.109400    2559 generic.go:356] "Generic (PLEG): container finished" podID="87646093-daaf-4f94-9518-01e0dc4ebe82" containerID="d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507" exitCode=137
Sep 09 13:45:29.109828 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.109469    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:29.109828 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.109500    2559 scope.go:117] "RemoveContainer" containerID="d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507"
Sep 09 13:45:29.109828 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:29.109790    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7937-eaf3c1" container="test-curl-7937-eaf3c1"
Sep 09 13:45:29.119492 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.119475    2559 scope.go:117] "RemoveContainer" containerID="d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507"
Sep 09 13:45:29.119794 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:29.119776    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507\": container with ID starting with d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507 not found: ID does not exist" containerID="d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507"
Sep 09 13:45:29.119883 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.119807    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507"} err="failed to get container status \"d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507\": rpc error: code = NotFound desc = could not find container \"d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507\": container with ID starting with d4163d6565ec2dfff730dcf2d511bcacda38bd707cbfdb676ade1881f0800507 not found: ID does not exist"
Sep 09 13:45:29.121156 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:29.121134    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7937-4e1d8b\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" pod="opendatahub/test-curl-7937-4e1d8b"
Sep 09 13:45:29.157393 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.157373    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:29.432283 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:45:29.432254    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2687c673_89bf_4b1d_bf0d_6980f7e65485.slice/crio-d5ad4ee300a3c88dc8d1e94c2ba6c6339615e1ae09f9e2d71e227c880c403372 WatchSource:0}: Error finding container d5ad4ee300a3c88dc8d1e94c2ba6c6339615e1ae09f9e2d71e227c880c403372: Status 404 returned error can't find the container with id d5ad4ee300a3c88dc8d1e94c2ba6c6339615e1ae09f9e2d71e227c880c403372
Sep 09 13:45:29.432991 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.432968    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7937-948786"]
Sep 09 13:45:29.619886 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:29.619854    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87646093-daaf-4f94-9518-01e0dc4ebe82" path="/var/lib/kubelet/pods/87646093-daaf-4f94-9518-01e0dc4ebe82/volumes"
Sep 09 13:45:30.118276 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.118250    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-948786" event={"ID":"2687c673-89bf-4b1d-bf0d-6980f7e65485","Type":"ContainerStarted","Data":"16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5"}
Sep 09 13:45:30.118618 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.118295    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7937-948786" event={"ID":"2687c673-89bf-4b1d-bf0d-6980f7e65485","Type":"ContainerStarted","Data":"d5ad4ee300a3c88dc8d1e94c2ba6c6339615e1ae09f9e2d71e227c880c403372"}
Sep 09 13:45:30.118618 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.118379    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7937-eaf3c1" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" containerName="test-curl-7937-eaf3c1" containerID="cri-o://0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a" gracePeriod=2
Sep 09 13:45:30.135299 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.135256    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7937-948786" podStartSLOduration=2.135241832 podStartE2EDuration="2.135241832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:45:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:30.134408139 +0000 UTC m=+1149.023108130" watchObservedRunningTime="2026-09-09 13:45:30.135241832 +0000 UTC m=+1149.023941842"
Sep 09 13:45:30.287786 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.287766    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:30.441800 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.441744    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdsbk\" (UniqueName: \"kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk\") pod \"0887a094-3f9e-4bae-87c6-26aac1bf1773\" (UID: \"0887a094-3f9e-4bae-87c6-26aac1bf1773\") "
Sep 09 13:45:30.443493 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.443469    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk" (OuterVolumeSpecName: "kube-api-access-hdsbk") pod "0887a094-3f9e-4bae-87c6-26aac1bf1773" (UID: "0887a094-3f9e-4bae-87c6-26aac1bf1773"). InnerVolumeSpecName "kube-api-access-hdsbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:45:30.543108 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.543082    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdsbk\" (UniqueName: \"kubernetes.io/projected/0887a094-3f9e-4bae-87c6-26aac1bf1773-kube-api-access-hdsbk\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:45:30.743138 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.743104    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7937-948786"]
Sep 09 13:45:30.749751 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:30.749716    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7937-948786"]
Sep 09 13:45:31.126600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.126504    2559 generic.go:356] "Generic (PLEG): container finished" podID="0887a094-3f9e-4bae-87c6-26aac1bf1773" containerID="007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f" exitCode=137
Sep 09 13:45:31.126600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.126547    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:31.127132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.126633    2559 scope.go:117] "RemoveContainer" containerID="007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f"
Sep 09 13:45:31.127132 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:31.126850    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7937-948786" container="test-curl-7937-948786"
Sep 09 13:45:31.136970 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.136952    2559 scope.go:117] "RemoveContainer" containerID="007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f"
Sep 09 13:45:31.137251 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:31.137235    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f\": container with ID starting with 007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f not found: ID does not exist" containerID="007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f"
Sep 09 13:45:31.137298 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.137261    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f"} err="failed to get container status \"007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f\": rpc error: code = NotFound desc = could not find container \"007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f\": container with ID starting with 007a312415d6cffec45fbccfcfa083209f896dfe49764946d2f922c2b6b2689f not found: ID does not exist"
Sep 09 13:45:31.138369 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:31.138348    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7937-cc02cb\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:31.616018 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:31.615982    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7937-cc02cb\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" pod="opendatahub/test-curl-7937-cc02cb"
Sep 09 13:45:31.618835 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:31.618808    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0887a094-3f9e-4bae-87c6-26aac1bf1773" path="/var/lib/kubelet/pods/0887a094-3f9e-4bae-87c6-26aac1bf1773/volumes"
Sep 09 13:45:32.134913 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:32.134883    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7937-948786" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" containerName="test-curl-7937-948786" containerID="cri-o://16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5" gracePeriod=2
Sep 09 13:45:32.295841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:32.295818    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:32.462296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:32.462267    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcbqm\" (UniqueName: \"kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm\") pod \"4167e1c6-feb3-492d-9da7-bd42ab504399\" (UID: \"4167e1c6-feb3-492d-9da7-bd42ab504399\") "
Sep 09 13:45:32.464329 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:32.464301    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm" (OuterVolumeSpecName: "kube-api-access-xcbqm") pod "4167e1c6-feb3-492d-9da7-bd42ab504399" (UID: "4167e1c6-feb3-492d-9da7-bd42ab504399"). InnerVolumeSpecName "kube-api-access-xcbqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:45:32.563055 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:32.563030    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcbqm\" (UniqueName: \"kubernetes.io/projected/4167e1c6-feb3-492d-9da7-bd42ab504399-kube-api-access-xcbqm\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:45:33.142288 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.142249    2559 generic.go:356] "Generic (PLEG): container finished" podID="4167e1c6-feb3-492d-9da7-bd42ab504399" containerID="0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a" exitCode=137
Sep 09 13:45:33.142702 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.142303    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:33.142702 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.142353    2559 scope.go:117] "RemoveContainer" containerID="0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a"
Sep 09 13:45:33.152420 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.152403    2559 scope.go:117] "RemoveContainer" containerID="0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a"
Sep 09 13:45:33.152742 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:33.152719    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a\": container with ID starting with 0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a not found: ID does not exist" containerID="0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a"
Sep 09 13:45:33.152827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.152748    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a"} err="failed to get container status \"0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a\": rpc error: code = NotFound desc = could not find container \"0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a\": container with ID starting with 0ae683beaa52eef17ae9191c61142cd26eafc888e5e2938899b030a93301349a not found: ID does not exist"
Sep 09 13:45:33.154093 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:33.154068    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7937-eaf3c1\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" pod="opendatahub/test-curl-7937-eaf3c1"
Sep 09 13:45:33.617203 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:33.617167    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4167e1c6-feb3-492d-9da7-bd42ab504399" path="/var/lib/kubelet/pods/4167e1c6-feb3-492d-9da7-bd42ab504399/volumes"
Sep 09 13:45:34.305663 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:34.305634    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:34.482039 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:34.482014    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrcfx\" (UniqueName: \"kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx\") pod \"2687c673-89bf-4b1d-bf0d-6980f7e65485\" (UID: \"2687c673-89bf-4b1d-bf0d-6980f7e65485\") "
Sep 09 13:45:34.483970 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:34.483947    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx" (OuterVolumeSpecName: "kube-api-access-wrcfx") pod "2687c673-89bf-4b1d-bf0d-6980f7e65485" (UID: "2687c673-89bf-4b1d-bf0d-6980f7e65485"). InnerVolumeSpecName "kube-api-access-wrcfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:45:34.583375 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:34.583347    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrcfx\" (UniqueName: \"kubernetes.io/projected/2687c673-89bf-4b1d-bf0d-6980f7e65485-kube-api-access-wrcfx\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:45:35.156302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.156261    2559 generic.go:356] "Generic (PLEG): container finished" podID="2687c673-89bf-4b1d-bf0d-6980f7e65485" containerID="16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5" exitCode=137
Sep 09 13:45:35.156491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.156309    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:35.156491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.156366    2559 scope.go:117] "RemoveContainer" containerID="16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5"
Sep 09 13:45:35.165620 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.165424    2559 scope.go:117] "RemoveContainer" containerID="16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5"
Sep 09 13:45:35.165718 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:35.165698    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5\": container with ID starting with 16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5 not found: ID does not exist" containerID="16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5"
Sep 09 13:45:35.165761 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.165728    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5"} err="failed to get container status \"16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5\": rpc error: code = NotFound desc = could not find container \"16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5\": container with ID starting with 16d15a75e710520616c12707c6b3bd60b13ab8aec44c5cfeb0cc0f22948b97a5 not found: ID does not exist"
Sep 09 13:45:35.166215 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:45:35.166195    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7937-948786\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" pod="opendatahub/test-curl-7937-948786"
Sep 09 13:45:35.617838 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.617804    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" path="/var/lib/kubelet/pods/2687c673-89bf-4b1d-bf0d-6980f7e65485/volumes"
Sep 09 13:45:35.971467 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.971437    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:45:35.972131 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.972110    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" containerName="test-curl-7937-948786"
Sep 09 13:45:35.972131 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.972131    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" containerName="test-curl-7937-948786"
Sep 09 13:45:35.972306 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.972272    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="2687c673-89bf-4b1d-bf0d-6980f7e65485" containerName="test-curl-7937-948786"
Sep 09 13:45:35.977223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.977200    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:35.982067 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.982048    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-638c40f1-serving-cert\""
Sep 09 13:45:35.984015 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:35.983993    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:45:36.098471 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.098437    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.098471 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.098473    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.098720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.098525    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.098720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.098625    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r227w\" (UniqueName: \"kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.199530 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.199496    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.199752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.199543    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.199752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.199623    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.199752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.199652    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r227w\" (UniqueName: \"kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.200296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.200272    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.202083 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.202063    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.202083 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.202074    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.208277 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.208253    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r227w\" (UniqueName: \"kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w\") pod \"maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.292818 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.292751    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:36.573105 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:36.573078    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:45:36.574514 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:45:36.574487    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866e7c04_f629_44bb_8cf7_441e4f022ea1.slice/crio-13787ead3ec0411fd46bb407e629692e171ceee7ada76c569f13c63a491c03db WatchSource:0}: Error finding container 13787ead3ec0411fd46bb407e629692e171ceee7ada76c569f13c63a491c03db: Status 404 returned error can't find the container with id 13787ead3ec0411fd46bb407e629692e171ceee7ada76c569f13c63a491c03db
Sep 09 13:45:37.171619 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:37.171544    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" event={"ID":"866e7c04-f629-44bb-8cf7-441e4f022ea1","Type":"ContainerStarted","Data":"13787ead3ec0411fd46bb407e629692e171ceee7ada76c569f13c63a491c03db"}
Sep 09 13:45:38.178917 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:38.178872    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" event={"ID":"866e7c04-f629-44bb-8cf7-441e4f022ea1","Type":"ContainerStarted","Data":"865bb7607a18051c9deb5c598fb5a77aa2e6fadc2af5b4b13a726580f83cf3ec"}
Sep 09 13:45:38.179528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:38.178978    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:45:38.200682 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:38.200631    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" podStartSLOduration=2.592870443 podStartE2EDuration="3.200616712s" totalImagesPullingTime="607.746269ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:45:38.1978874 +0000 UTC m=+1157.086587383" watchObservedRunningTime="2026-09-09 13:45:38.200616712 +0000 UTC m=+1157.089316702"
Sep 09 13:45:44.190099 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:45:44.190016    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:46:01.702166 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.702132    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:46:01.705995 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.705974    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.710877 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.710853    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-638c40f1\"/\"openshift-service-ca.crt\""
Sep 09 13:46:01.711032 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.710898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-638c40f1\"/\"default-dockercfg-t4czj\""
Sep 09 13:46:01.711032 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.710898    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-638c40f1\"/\"e2e-lifecycle-model-638c40f1-kserve-self-signed-certs\""
Sep 09 13:46:01.711229 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.710903    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-638c40f1\"/\"kube-root-ca.crt\""
Sep 09 13:46:01.733933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.733906    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:46:01.829406 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829350    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.829611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829415    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ls7\" (UniqueName: \"kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.829611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829530    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.829611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829566    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.829793 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829632    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.829793 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.829661    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930179 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930138    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930361 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930186    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930361 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930223    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930361 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930253    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930361 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930349    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930646 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930408    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ls7\" (UniqueName: \"kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930646 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930552    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930646 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930632    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.930827 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.930769    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.932939 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.932912    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.933135 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.933108    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:01.939463 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:01.939442    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ls7\" (UniqueName: \"kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7\") pod \"e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") " pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:02.018380 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:02.018352    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:02.295252 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:02.295222    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:46:02.296269 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:46:02.296241    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48591a0d_55b6_47de_ae81_2f75d8ee3f5c.slice/crio-1036a9ef6ef693a21d60dfa9db6be7fdadb1c19e59e6b2130b149c666cc6e31c WatchSource:0}: Error finding container 1036a9ef6ef693a21d60dfa9db6be7fdadb1c19e59e6b2130b149c666cc6e31c: Status 404 returned error can't find the container with id 1036a9ef6ef693a21d60dfa9db6be7fdadb1c19e59e6b2130b149c666cc6e31c
Sep 09 13:46:02.330342 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:02.330310    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" event={"ID":"48591a0d-55b6-47de-ae81-2f75d8ee3f5c","Type":"ContainerStarted","Data":"1036a9ef6ef693a21d60dfa9db6be7fdadb1c19e59e6b2130b149c666cc6e31c"}
Sep 09 13:46:03.339258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:03.339222    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" event={"ID":"48591a0d-55b6-47de-ae81-2f75d8ee3f5c","Type":"ContainerStarted","Data":"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"}
Sep 09 13:46:03.357604 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:03.357538    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" podStartSLOduration=2.357524081 podStartE2EDuration="2.357524081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:46:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:46:03.355852459 +0000 UTC m=+1182.244552446" watchObservedRunningTime="2026-09-09 13:46:03.357524081 +0000 UTC m=+1182.246224071"
Sep 09 13:46:12.018849 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:12.018811    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:12.019351 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:12.018864    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:12.031531 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:12.031509    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:12.403948 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:12.403855    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:46:44.471105 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.471070    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:46:44.471759 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.471406    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" podUID="866e7c04-f629-44bb-8cf7-441e4f022ea1" containerName="maas-api" containerID="cri-o://865bb7607a18051c9deb5c598fb5a77aa2e6fadc2af5b4b13a726580f83cf3ec" gracePeriod=30
Sep 09 13:46:44.604525 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.604482    2559 generic.go:356] "Generic (PLEG): container finished" podID="866e7c04-f629-44bb-8cf7-441e4f022ea1" containerID="865bb7607a18051c9deb5c598fb5a77aa2e6fadc2af5b4b13a726580f83cf3ec" exitCode=0
Sep 09 13:46:44.604775 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.604526    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" event={"ID":"866e7c04-f629-44bb-8cf7-441e4f022ea1","Type":"ContainerDied","Data":"865bb7607a18051c9deb5c598fb5a77aa2e6fadc2af5b4b13a726580f83cf3ec"}
Sep 09 13:46:44.727141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.727083    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:46:44.814955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.814927    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs\") pod \"866e7c04-f629-44bb-8cf7-441e4f022ea1\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") "
Sep 09 13:46:44.815101 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.815032    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r227w\" (UniqueName: \"kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w\") pod \"866e7c04-f629-44bb-8cf7-441e4f022ea1\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") "
Sep 09 13:46:44.815101 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.815058    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca\") pod \"866e7c04-f629-44bb-8cf7-441e4f022ea1\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") "
Sep 09 13:46:44.815189 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.815111    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls\") pod \"866e7c04-f629-44bb-8cf7-441e4f022ea1\" (UID: \"866e7c04-f629-44bb-8cf7-441e4f022ea1\") "
Sep 09 13:46:44.815532 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.815499    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "866e7c04-f629-44bb-8cf7-441e4f022ea1" (UID: "866e7c04-f629-44bb-8cf7-441e4f022ea1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:46:44.816987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.816959    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "866e7c04-f629-44bb-8cf7-441e4f022ea1" (UID: "866e7c04-f629-44bb-8cf7-441e4f022ea1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:46:44.817160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.817139    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "866e7c04-f629-44bb-8cf7-441e4f022ea1" (UID: "866e7c04-f629-44bb-8cf7-441e4f022ea1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:46:44.817461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.817443    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w" (OuterVolumeSpecName: "kube-api-access-r227w") pod "866e7c04-f629-44bb-8cf7-441e4f022ea1" (UID: "866e7c04-f629-44bb-8cf7-441e4f022ea1"). InnerVolumeSpecName "kube-api-access-r227w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:46:44.916600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.916545    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/866e7c04-f629-44bb-8cf7-441e4f022ea1-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:46:44.916600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.916583    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:46:44.916600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.916594    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/866e7c04-f629-44bb-8cf7-441e4f022ea1-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:46:44.916600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:44.916602    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r227w\" (UniqueName: \"kubernetes.io/projected/866e7c04-f629-44bb-8cf7-441e4f022ea1-kube-api-access-r227w\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:46:45.614063 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:45.614031    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"
Sep 09 13:46:45.617771 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:45.617737    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg" event={"ID":"866e7c04-f629-44bb-8cf7-441e4f022ea1","Type":"ContainerDied","Data":"13787ead3ec0411fd46bb407e629692e171ceee7ada76c569f13c63a491c03db"}
Sep 09 13:46:45.617925 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:45.617788    2559 scope.go:117] "RemoveContainer" containerID="865bb7607a18051c9deb5c598fb5a77aa2e6fadc2af5b4b13a726580f83cf3ec"
Sep 09 13:46:45.640403 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:45.640379    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:46:45.644731 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:45.644709    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-638c40f1-7ddc65f86d-9zgkg"]
Sep 09 13:46:47.618890 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:47.618856    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="866e7c04-f629-44bb-8cf7-441e4f022ea1" path="/var/lib/kubelet/pods/866e7c04-f629-44bb-8cf7-441e4f022ea1/volumes"
Sep 09 13:46:52.751461 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:46:52.751421    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:47:07.771446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:07.771379    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" podUID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" containerName="main" containerID="cri-o://abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5" gracePeriod=45
Sep 09 13:47:08.028515 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.028450    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:47:08.125669 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125633    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.125858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125677    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.125858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125702    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.125858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125743    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4ls7\" (UniqueName: \"kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.125858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125772    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.125858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125799    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir\") pod \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\" (UID: \"48591a0d-55b6-47de-ae81-2f75d8ee3f5c\") "
Sep 09 13:47:08.126121 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.125968    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache" (OuterVolumeSpecName: "model-cache") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:47:08.126121 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.126016    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home" (OuterVolumeSpecName: "home") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:47:08.126121 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.126101    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:47:08.126266 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.126194    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-model-cache\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.126266 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.126215    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-home\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.126266 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.126254    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tmp-dir\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.127802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.127781    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm" (OuterVolumeSpecName: "dshm") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:47:08.127916 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.127894    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7" (OuterVolumeSpecName: "kube-api-access-l4ls7") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "kube-api-access-l4ls7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:47:08.127968 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.127899    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "48591a0d-55b6-47de-ae81-2f75d8ee3f5c" (UID: "48591a0d-55b6-47de-ae81-2f75d8ee3f5c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:47:08.226960 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.226933    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4ls7\" (UniqueName: \"kubernetes.io/projected/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-kube-api-access-l4ls7\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.226960 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.226957    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-tls-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.227113 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.226967    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48591a0d-55b6-47de-ae81-2f75d8ee3f5c-dshm\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:08.758366 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.758334    2559 generic.go:356] "Generic (PLEG): container finished" podID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" containerID="abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5" exitCode=0
Sep 09 13:47:08.758535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.758377    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"
Sep 09 13:47:08.758535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.758418    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" event={"ID":"48591a0d-55b6-47de-ae81-2f75d8ee3f5c","Type":"ContainerDied","Data":"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"}
Sep 09 13:47:08.758535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.758465    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7" event={"ID":"48591a0d-55b6-47de-ae81-2f75d8ee3f5c","Type":"ContainerDied","Data":"1036a9ef6ef693a21d60dfa9db6be7fdadb1c19e59e6b2130b149c666cc6e31c"}
Sep 09 13:47:08.758535 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.758488    2559 scope.go:117] "RemoveContainer" containerID="abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"
Sep 09 13:47:08.768505 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.768488    2559 scope.go:117] "RemoveContainer" containerID="abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"
Sep 09 13:47:08.768795 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:47:08.768779    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5\": container with ID starting with abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5 not found: ID does not exist" containerID="abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"
Sep 09 13:47:08.768848 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.768802    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5"} err="failed to get container status \"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5\": rpc error: code = NotFound desc = could not find container \"abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5\": container with ID starting with abba9990e448efafd03db36c6e9f28a091eb6246d31ca1152d9d31c25e6984d5 not found: ID does not exist"
Sep 09 13:47:08.787006 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.786985    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:47:08.789455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:08.789436    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-638c40f1/e2e-lifecycle-model-638c40f1-kserve-8446cd9dd7-krzb7"]
Sep 09 13:47:09.617774 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:09.617737    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" path="/var/lib/kubelet/pods/48591a0d-55b6-47de-ae81-2f75d8ee3f5c/volumes"
Sep 09 13:47:26.679223 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679176    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7940-d66c04"]
Sep 09 13:47:26.679683 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679660    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="866e7c04-f629-44bb-8cf7-441e4f022ea1" containerName="maas-api"
Sep 09 13:47:26.679683 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679674    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="866e7c04-f629-44bb-8cf7-441e4f022ea1" containerName="maas-api"
Sep 09 13:47:26.679764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679687    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" containerName="main"
Sep 09 13:47:26.679764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679693    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" containerName="main"
Sep 09 13:47:26.679830 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679765    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="48591a0d-55b6-47de-ae81-2f75d8ee3f5c" containerName="main"
Sep 09 13:47:26.679830 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.679775    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="866e7c04-f629-44bb-8cf7-441e4f022ea1" containerName="maas-api"
Sep 09 13:47:26.684254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.684235    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:26.687233 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.687204    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qg8jq\""
Sep 09 13:47:26.688114 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.688083    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-d66c04"]
Sep 09 13:47:26.797900 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.797864    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw29b\" (UniqueName: \"kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b\") pod \"test-curl-7940-d66c04\" (UID: \"12390140-36c9-4681-8cff-05e69ef8ab72\") " pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:26.899059 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.899020    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw29b\" (UniqueName: \"kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b\") pod \"test-curl-7940-d66c04\" (UID: \"12390140-36c9-4681-8cff-05e69ef8ab72\") " pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:26.908860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.908831    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw29b\" (UniqueName: \"kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b\") pod \"test-curl-7940-d66c04\" (UID: \"12390140-36c9-4681-8cff-05e69ef8ab72\") " pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:26.994723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:26.994688    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:27.269997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:27.269973    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-d66c04"]
Sep 09 13:47:27.270918 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:47:27.270891    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod12390140_36c9_4681_8cff_05e69ef8ab72.slice/crio-229cb5b94bc2a9cdec951f7fc315fceaf56bdb56228e06cd994195b5fae93c7b WatchSource:0}: Error finding container 229cb5b94bc2a9cdec951f7fc315fceaf56bdb56228e06cd994195b5fae93c7b: Status 404 returned error can't find the container with id 229cb5b94bc2a9cdec951f7fc315fceaf56bdb56228e06cd994195b5fae93c7b
Sep 09 13:47:27.878417 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:27.878385    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-d66c04" event={"ID":"12390140-36c9-4681-8cff-05e69ef8ab72","Type":"ContainerStarted","Data":"ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52"}
Sep 09 13:47:27.878417 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:27.878427    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-d66c04" event={"ID":"12390140-36c9-4681-8cff-05e69ef8ab72","Type":"ContainerStarted","Data":"229cb5b94bc2a9cdec951f7fc315fceaf56bdb56228e06cd994195b5fae93c7b"}
Sep 09 13:47:27.910810 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:27.910752    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7940-d66c04" podStartSLOduration=1.910730816 podStartE2EDuration="1.910730816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:47:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:47:27.908246206 +0000 UTC m=+1266.796946208" watchObservedRunningTime="2026-09-09 13:47:27.910730816 +0000 UTC m=+1266.799430803"
Sep 09 13:47:28.610997 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:28.610909    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7940-d66c04"]
Sep 09 13:47:28.623929 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:28.623890    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7940-d66c04"]
Sep 09 13:47:28.883945 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:47:28.883874    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7940-d66c04" container="test-curl-7940-d66c04"
Sep 09 13:47:29.889365 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:29.889318    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7940-d66c04" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" containerName="test-curl-7940-d66c04" containerID="cri-o://ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52" gracePeriod=2
Sep 09 13:47:32.069356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.069331    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:32.148215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.148145    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw29b\" (UniqueName: \"kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b\") pod \"12390140-36c9-4681-8cff-05e69ef8ab72\" (UID: \"12390140-36c9-4681-8cff-05e69ef8ab72\") "
Sep 09 13:47:32.150044 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.150017    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b" (OuterVolumeSpecName: "kube-api-access-qw29b") pod "12390140-36c9-4681-8cff-05e69ef8ab72" (UID: "12390140-36c9-4681-8cff-05e69ef8ab72"). InnerVolumeSpecName "kube-api-access-qw29b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:47:32.249750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.249718    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw29b\" (UniqueName: \"kubernetes.io/projected/12390140-36c9-4681-8cff-05e69ef8ab72-kube-api-access-qw29b\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:47:32.921876 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.921842    2559 generic.go:356] "Generic (PLEG): container finished" podID="12390140-36c9-4681-8cff-05e69ef8ab72" containerID="ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52" exitCode=137
Sep 09 13:47:32.922045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.921882    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:32.922045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.921929    2559 scope.go:117] "RemoveContainer" containerID="ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52"
Sep 09 13:47:32.931117 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.931104    2559 scope.go:117] "RemoveContainer" containerID="ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52"
Sep 09 13:47:32.931357 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:47:32.931338    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52\": container with ID starting with ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52 not found: ID does not exist" containerID="ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52"
Sep 09 13:47:32.931408 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:32.931365    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52"} err="failed to get container status \"ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52\": rpc error: code = NotFound desc = could not find container \"ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52\": container with ID starting with ff2052df21d761cd25399ee3516b0a1bf2a1dc491c9f3602f12ca11b2196ed52 not found: ID does not exist"
Sep 09 13:47:32.932749 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:47:32.932730    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7940-d66c04\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" pod="opendatahub/test-curl-7940-d66c04"
Sep 09 13:47:33.618582 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:33.618548    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" path="/var/lib/kubelet/pods/12390140-36c9-4681-8cff-05e69ef8ab72/volumes"
Sep 09 13:47:59.377630 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.377596    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:47:59.378167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.378036    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" containerName="test-curl-7940-d66c04"
Sep 09 13:47:59.378167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.378048    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" containerName="test-curl-7940-d66c04"
Sep 09 13:47:59.378167 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.378131    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="12390140-36c9-4681-8cff-05e69ef8ab72" containerName="test-curl-7940-d66c04"
Sep 09 13:47:59.381308 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.381287    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.385455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.385432    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8f2df2\"/\"kube-root-ca.crt\""
Sep 09 13:47:59.385602 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.385435    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8f2df2\"/\"sub-test-model-8f2df2-kserve-self-signed-certs\""
Sep 09 13:47:59.385602 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.385436    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-8f2df2\"/\"openshift-service-ca.crt\""
Sep 09 13:47:59.385602 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.385435    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-8f2df2\"/\"default-dockercfg-2fx29\""
Sep 09 13:47:59.395256 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.395231    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:47:59.484507 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484470    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.484682 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484520    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.484682 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484540    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.484682 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484631    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.484802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484710    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99ptd\" (UniqueName: \"kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.484802 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.484736    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.585898 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.585863    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99ptd\" (UniqueName: \"kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.585903    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.585952    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.585998    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.586025    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.586064    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586417 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.586389    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586494 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.586406    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.586494 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.586469    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.588455 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.588424    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.588607 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.588588    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.599328 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.599297    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99ptd\" (UniqueName: \"kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd\") pod \"sub-test-model-8f2df2-kserve-797698b9cf-62489\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") " pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.692161 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.692100    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:47:59.966438 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:47:59.966411    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:47:59.967276 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:47:59.967252    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdc4e5cf_f0e9_4dea_b62d_066db153141a.slice/crio-9935ff91bd03c9a7999d655922283773fa1bbfdcb757c4f00b9f16fd1e1a4c26 WatchSource:0}: Error finding container 9935ff91bd03c9a7999d655922283773fa1bbfdcb757c4f00b9f16fd1e1a4c26: Status 404 returned error can't find the container with id 9935ff91bd03c9a7999d655922283773fa1bbfdcb757c4f00b9f16fd1e1a4c26
Sep 09 13:48:00.113192 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:00.113155    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" event={"ID":"bdc4e5cf-f0e9-4dea-b62d-066db153141a","Type":"ContainerStarted","Data":"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"}
Sep 09 13:48:00.113411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:00.113383    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" event={"ID":"bdc4e5cf-f0e9-4dea-b62d-066db153141a","Type":"ContainerStarted","Data":"9935ff91bd03c9a7999d655922283773fa1bbfdcb757c4f00b9f16fd1e1a4c26"}
Sep 09 13:48:00.137426 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:00.137376    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" podStartSLOduration=1.137359935 podStartE2EDuration="1.137359935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:47:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:48:00.136584163 +0000 UTC m=+1299.025284151" watchObservedRunningTime="2026-09-09 13:48:00.137359935 +0000 UTC m=+1299.026059925"
Sep 09 13:48:09.692678 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:09.692645    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:48:09.693116 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:09.692691    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:48:09.705603 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:09.705559    2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:48:10.190410 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:10.190384    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:48:49.958215 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.958129    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:48:49.964392 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.964365    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:49.966905 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.966881    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-464c54-serving-cert\""
Sep 09 13:48:49.975805 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.975780    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:48:49.984098 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.984073    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:49.984217 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.984108    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:49.984267 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.984213    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r6g5\" (UniqueName: \"kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:49.984310 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:49.984265    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.085241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.085206    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r6g5\" (UniqueName: \"kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.085425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.085254    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.085425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.085308    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.085425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.085323    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.086006 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.085981    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.087804 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.087782    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.087895 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.087820    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.098412 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.098391    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r6g5\" (UniqueName: \"kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5\") pod \"maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.276808 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.276724    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:50.556434 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.556409    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:48:50.557202 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:48:50.557179    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41ba6f5_4549_425a_a705_5f09318e34c5.slice/crio-02675148acab5e1c0328c34c786343abb8e71503c0d16f273b04705e5a64937a WatchSource:0}: Error finding container 02675148acab5e1c0328c34c786343abb8e71503c0d16f273b04705e5a64937a: Status 404 returned error can't find the container with id 02675148acab5e1c0328c34c786343abb8e71503c0d16f273b04705e5a64937a
Sep 09 13:48:50.558520 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:50.558502    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:48:51.450707 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:51.450669    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" event={"ID":"c41ba6f5-4549-425a-a705-5f09318e34c5","Type":"ContainerStarted","Data":"02675148acab5e1c0328c34c786343abb8e71503c0d16f273b04705e5a64937a"}
Sep 09 13:48:52.458642 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:52.458603    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" event={"ID":"c41ba6f5-4549-425a-a705-5f09318e34c5","Type":"ContainerStarted","Data":"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"}
Sep 09 13:48:52.459111 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:52.458788    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:48:52.477129 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:52.477084    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" podStartSLOduration=2.336974464 podStartE2EDuration="3.477072194s" totalImagesPullingTime="1.14009773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:48:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:48:52.47581191 +0000 UTC m=+1351.364511896" watchObservedRunningTime="2026-09-09 13:48:52.477072194 +0000 UTC m=+1351.365772183"
Sep 09 13:48:58.470356 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:48:58.470326    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:49:05.221660 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.221618    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7940-ca19c7"]
Sep 09 13:49:05.225708 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.225684    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:05.228244 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.228225    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qg8jq\""
Sep 09 13:49:05.233933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.233908    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-ca19c7"]
Sep 09 13:49:05.328391 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.328345    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh8h2\" (UniqueName: \"kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2\") pod \"test-curl-7940-ca19c7\" (UID: \"7379082f-731a-45bf-9cef-91df31807a18\") " pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:05.429829 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.429795    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh8h2\" (UniqueName: \"kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2\") pod \"test-curl-7940-ca19c7\" (UID: \"7379082f-731a-45bf-9cef-91df31807a18\") " pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:05.438431 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.438403    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh8h2\" (UniqueName: \"kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2\") pod \"test-curl-7940-ca19c7\" (UID: \"7379082f-731a-45bf-9cef-91df31807a18\") " pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:05.536921 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.536854    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:05.809163 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:05.809125    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-ca19c7"]
Sep 09 13:49:05.810584 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:49:05.810550    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7379082f_731a_45bf_9cef_91df31807a18.slice/crio-c2f0c7ea69fbe09cad4c24b724df2f78bb25ecc19fd24121559477587dae2a84 WatchSource:0}: Error finding container c2f0c7ea69fbe09cad4c24b724df2f78bb25ecc19fd24121559477587dae2a84: Status 404 returned error can't find the container with id c2f0c7ea69fbe09cad4c24b724df2f78bb25ecc19fd24121559477587dae2a84
Sep 09 13:49:06.544627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:06.544589    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-ca19c7" event={"ID":"7379082f-731a-45bf-9cef-91df31807a18","Type":"ContainerStarted","Data":"6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba"}
Sep 09 13:49:06.544627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:06.544634    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-ca19c7" event={"ID":"7379082f-731a-45bf-9cef-91df31807a18","Type":"ContainerStarted","Data":"c2f0c7ea69fbe09cad4c24b724df2f78bb25ecc19fd24121559477587dae2a84"}
Sep 09 13:49:06.561261 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:06.561214    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7940-ca19c7" podStartSLOduration=1.561192739 podStartE2EDuration="1.561192739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:49:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:49:06.558765871 +0000 UTC m=+1365.447465864" watchObservedRunningTime="2026-09-09 13:49:06.561192739 +0000 UTC m=+1365.449892729"
Sep 09 13:49:07.148227 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.148186    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7940-ca19c7"]
Sep 09 13:49:07.152980 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.152953    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7940-ca19c7"]
Sep 09 13:49:07.279819 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.279785    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7940-efd54e"]
Sep 09 13:49:07.280296 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.280282    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7379082f-731a-45bf-9cef-91df31807a18" containerName="test-curl-7940-ca19c7"
Sep 09 13:49:07.280345 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.280298    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7379082f-731a-45bf-9cef-91df31807a18" containerName="test-curl-7940-ca19c7"
Sep 09 13:49:07.280388 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.280379    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7379082f-731a-45bf-9cef-91df31807a18" containerName="test-curl-7940-ca19c7"
Sep 09 13:49:07.283759 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.283742    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:07.288778 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.288751    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-efd54e"]
Sep 09 13:49:07.349183 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.349147    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2cwn\" (UniqueName: \"kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn\") pod \"test-curl-7940-efd54e\" (UID: \"b1f0a068-ad9b-4411-a63e-d9a49336d9bb\") " pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:07.450145 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.450102    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2cwn\" (UniqueName: \"kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn\") pod \"test-curl-7940-efd54e\" (UID: \"b1f0a068-ad9b-4411-a63e-d9a49336d9bb\") " pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:07.459750 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.459718    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2cwn\" (UniqueName: \"kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn\") pod \"test-curl-7940-efd54e\" (UID: \"b1f0a068-ad9b-4411-a63e-d9a49336d9bb\") " pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:07.550990 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:07.550910    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7940-ca19c7" container="test-curl-7940-ca19c7"
Sep 09 13:49:07.594637 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.594614    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:07.866390 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:07.866363    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7940-efd54e"]
Sep 09 13:49:07.867487 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:49:07.867459    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb1f0a068_ad9b_4411_a63e_d9a49336d9bb.slice/crio-91978ceca2e3b162e32d7d9b55b10da7a39a0cf2ef9f304b28b3db7511c985e0 WatchSource:0}: Error finding container 91978ceca2e3b162e32d7d9b55b10da7a39a0cf2ef9f304b28b3db7511c985e0: Status 404 returned error can't find the container with id 91978ceca2e3b162e32d7d9b55b10da7a39a0cf2ef9f304b28b3db7511c985e0
Sep 09 13:49:08.558460 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:08.558423    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-efd54e" event={"ID":"b1f0a068-ad9b-4411-a63e-d9a49336d9bb","Type":"ContainerStarted","Data":"3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867"}
Sep 09 13:49:08.558869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:08.558469    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7940-efd54e" event={"ID":"b1f0a068-ad9b-4411-a63e-d9a49336d9bb","Type":"ContainerStarted","Data":"91978ceca2e3b162e32d7d9b55b10da7a39a0cf2ef9f304b28b3db7511c985e0"}
Sep 09 13:49:08.558869 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:08.558488    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7940-ca19c7" podUID="7379082f-731a-45bf-9cef-91df31807a18" containerName="test-curl-7940-ca19c7" containerID="cri-o://6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba" gracePeriod=2
Sep 09 13:49:08.574330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:08.574286    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7940-efd54e" podStartSLOduration=1.5742721240000002 podStartE2EDuration="1.574272124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:49:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:49:08.57346183 +0000 UTC m=+1367.462161821" watchObservedRunningTime="2026-09-09 13:49:08.574272124 +0000 UTC m=+1367.462972114"
Sep 09 13:49:09.150861 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:09.150827    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7940-efd54e"]
Sep 09 13:49:09.156679 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:09.156652    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7940-efd54e"]
Sep 09 13:49:09.565422 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:09.565386    2559 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7940-efd54e" container="test-curl-7940-efd54e"
Sep 09 13:49:10.572622 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:10.572583    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7940-efd54e" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" containerName="test-curl-7940-efd54e" containerID="cri-o://3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867" gracePeriod=2
Sep 09 13:49:10.705815 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:10.705792    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:10.780554 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:10.780530    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh8h2\" (UniqueName: \"kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2\") pod \"7379082f-731a-45bf-9cef-91df31807a18\" (UID: \"7379082f-731a-45bf-9cef-91df31807a18\") "
Sep 09 13:49:10.782631 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:10.782603    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2" (OuterVolumeSpecName: "kube-api-access-xh8h2") pod "7379082f-731a-45bf-9cef-91df31807a18" (UID: "7379082f-731a-45bf-9cef-91df31807a18"). InnerVolumeSpecName "kube-api-access-xh8h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:49:10.882050 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:10.881980    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh8h2\" (UniqueName: \"kubernetes.io/projected/7379082f-731a-45bf-9cef-91df31807a18-kube-api-access-xh8h2\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:11.102258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.102224    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:49:11.580409 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.580374    2559 generic.go:356] "Generic (PLEG): container finished" podID="7379082f-731a-45bf-9cef-91df31807a18" containerID="6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba" exitCode=137
Sep 09 13:49:11.580824 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.580415    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:11.580824 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.580471    2559 scope.go:117] "RemoveContainer" containerID="6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba"
Sep 09 13:49:11.591521 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:11.591482    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7940-ca19c7\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="7379082f-731a-45bf-9cef-91df31807a18" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:11.594076 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.594056    2559 scope.go:117] "RemoveContainer" containerID="6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba"
Sep 09 13:49:11.594356 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:11.594338    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba\": container with ID starting with 6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba not found: ID does not exist" containerID="6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba"
Sep 09 13:49:11.594405 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.594366    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba"} err="failed to get container status \"6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba\": rpc error: code = NotFound desc = could not find container \"6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba\": container with ID starting with 6d5602cc92fe9311e3e78b74943dc06ecb8bf4317cb64a655f7246ad460d7dba not found: ID does not exist"
Sep 09 13:49:11.617807 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:11.617784    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7379082f-731a-45bf-9cef-91df31807a18" path="/var/lib/kubelet/pods/7379082f-731a-45bf-9cef-91df31807a18/volumes"
Sep 09 13:49:11.617958 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:11.617932    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7940-ca19c7\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="7379082f-731a-45bf-9cef-91df31807a18" pod="opendatahub/test-curl-7940-ca19c7"
Sep 09 13:49:12.763389 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:12.763363    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:12.900669 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:12.900605    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2cwn\" (UniqueName: \"kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn\") pod \"b1f0a068-ad9b-4411-a63e-d9a49336d9bb\" (UID: \"b1f0a068-ad9b-4411-a63e-d9a49336d9bb\") "
Sep 09 13:49:12.902548 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:12.902522    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn" (OuterVolumeSpecName: "kube-api-access-x2cwn") pod "b1f0a068-ad9b-4411-a63e-d9a49336d9bb" (UID: "b1f0a068-ad9b-4411-a63e-d9a49336d9bb"). InnerVolumeSpecName "kube-api-access-x2cwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:49:13.001634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.001602    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2cwn\" (UniqueName: \"kubernetes.io/projected/b1f0a068-ad9b-4411-a63e-d9a49336d9bb-kube-api-access-x2cwn\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:13.594845 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.594807    2559 generic.go:356] "Generic (PLEG): container finished" podID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" containerID="3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867" exitCode=137
Sep 09 13:49:13.595002 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.594851    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:13.595002 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.594875    2559 scope.go:117] "RemoveContainer" containerID="3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867"
Sep 09 13:49:13.604440 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.604418    2559 scope.go:117] "RemoveContainer" containerID="3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867"
Sep 09 13:49:13.604889 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:13.604862    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867\": container with ID starting with 3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867 not found: ID does not exist" containerID="3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867"
Sep 09 13:49:13.604966 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.604900    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867"} err="failed to get container status \"3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867\": rpc error: code = NotFound desc = could not find container \"3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867\": container with ID starting with 3a89a0bb662842e493725cb8453c3205be05b7a3319ffbb6ba4b77567b5f4867 not found: ID does not exist"
Sep 09 13:49:13.605245 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:13.605218    2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7940-efd54e\" is forbidden: User \"system:node:ip-10-0-136-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-35.ec2.internal' and this object" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" pod="opendatahub/test-curl-7940-efd54e"
Sep 09 13:49:13.618538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:13.618517    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" path="/var/lib/kubelet/pods/b1f0a068-ad9b-4411-a63e-d9a49336d9bb/volumes"
Sep 09 13:49:18.201735 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.201693    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:49:18.203528 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.203495    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" containerName="test-curl-7940-efd54e"
Sep 09 13:49:18.203684 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.203535    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" containerName="test-curl-7940-efd54e"
Sep 09 13:49:18.203749 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.203711    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1f0a068-ad9b-4411-a63e-d9a49336d9bb" containerName="test-curl-7940-efd54e"
Sep 09 13:49:18.217271 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.217238    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:49:18.217395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.217360    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.220232 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.220154    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-4bfe1d-serving-cert\""
Sep 09 13:49:18.352099 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.352062    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfh78\" (UniqueName: \"kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.352099 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.352101    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.352347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.352163    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.352347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.352195    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.453305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.453208    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfh78\" (UniqueName: \"kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.453305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.453264    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.453549 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.453311    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.453549 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.453351    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.453549 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:18.453426    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-serving-cert: secret "maas-api-e2e-api-b-4bfe1d-serving-cert" not found
Sep 09 13:49:18.453549 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:18.453511    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls podName:0e9dd4bc-a101-4400-90c0-b018cb17b637 nodeName:}" failed. No retries permitted until 2026-09-09 13:49:18.953489071 +0000 UTC m=+1377.842189041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls") pod "maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" (UID: "0e9dd4bc-a101-4400-90c0-b018cb17b637") : secret "maas-api-e2e-api-b-4bfe1d-serving-cert" not found
Sep 09 13:49:18.454132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.454106    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.455736 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.455717    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.462330 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.462308    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfh78\" (UniqueName: \"kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.958693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.958654    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:18.961199 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:18.961172    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") pod \"maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:19.129838 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:19.129795    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:19.413720 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:19.413691    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:49:19.415000 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:49:19.414973    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9dd4bc_a101_4400_90c0_b018cb17b637.slice/crio-a54603f6450d72af05511a8e4de6002522d775458f32cab785ab2808e8bd9b25 WatchSource:0}: Error finding container a54603f6450d72af05511a8e4de6002522d775458f32cab785ab2808e8bd9b25: Status 404 returned error can't find the container with id a54603f6450d72af05511a8e4de6002522d775458f32cab785ab2808e8bd9b25
Sep 09 13:49:19.637238 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:19.637203    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" event={"ID":"0e9dd4bc-a101-4400-90c0-b018cb17b637","Type":"ContainerStarted","Data":"a54603f6450d72af05511a8e4de6002522d775458f32cab785ab2808e8bd9b25"}
Sep 09 13:49:20.646193 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:20.646149    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" event={"ID":"0e9dd4bc-a101-4400-90c0-b018cb17b637","Type":"ContainerStarted","Data":"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"}
Sep 09 13:49:20.646737 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:20.646230    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:20.664956 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:20.664899    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" podStartSLOduration=2.20917104 podStartE2EDuration="2.664883176s" totalImagesPullingTime="455.712136ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:49:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:49:20.66275102 +0000 UTC m=+1379.551451006" watchObservedRunningTime="2026-09-09 13:49:20.664883176 +0000 UTC m=+1379.553583165"
Sep 09 13:49:26.126959 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.126901    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" podUID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" containerName="main" containerID="cri-o://c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01" gracePeriod=45
Sep 09 13:49:26.379477 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.379421    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:49:26.428557 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428526    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.428732 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428605    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.428732 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428696    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99ptd\" (UniqueName: \"kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.428732 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428727    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.428860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428760    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.428860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428821    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache" (OuterVolumeSpecName: "model-cache") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:49:26.428860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.428830    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm\") pod \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\" (UID: \"bdc4e5cf-f0e9-4dea-b62d-066db153141a\") "
Sep 09 13:49:26.429096 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.429073    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:49:26.429231 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.429214    2559 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-model-cache\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.429313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.429232    2559 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tmp-dir\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.429313 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.429234    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home" (OuterVolumeSpecName: "home") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:49:26.430870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.430846    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm" (OuterVolumeSpecName: "dshm") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:49:26.430963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.430898    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd" (OuterVolumeSpecName: "kube-api-access-99ptd") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "kube-api-access-99ptd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:49:26.430963 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.430914    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bdc4e5cf-f0e9-4dea-b62d-066db153141a" (UID: "bdc4e5cf-f0e9-4dea-b62d-066db153141a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:49:26.529873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.529844    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99ptd\" (UniqueName: \"kubernetes.io/projected/bdc4e5cf-f0e9-4dea-b62d-066db153141a-kube-api-access-99ptd\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.529873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.529867    2559 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-home\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.529873 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.529876    2559 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bdc4e5cf-f0e9-4dea-b62d-066db153141a-dshm\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.530037 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.529884    2559 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bdc4e5cf-f0e9-4dea-b62d-066db153141a-tls-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:49:26.656714 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.656651    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:49:26.684841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.684808    2559 generic.go:356] "Generic (PLEG): container finished" podID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" containerID="c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01" exitCode=0
Sep 09 13:49:26.684977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.684843    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"
Sep 09 13:49:26.684977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.684879    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" event={"ID":"bdc4e5cf-f0e9-4dea-b62d-066db153141a","Type":"ContainerDied","Data":"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"}
Sep 09 13:49:26.684977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.684956    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489" event={"ID":"bdc4e5cf-f0e9-4dea-b62d-066db153141a","Type":"ContainerDied","Data":"9935ff91bd03c9a7999d655922283773fa1bbfdcb757c4f00b9f16fd1e1a4c26"}
Sep 09 13:49:26.685148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.684982    2559 scope.go:117] "RemoveContainer" containerID="c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"
Sep 09 13:49:26.697718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.697695    2559 scope.go:117] "RemoveContainer" containerID="c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"
Sep 09 13:49:26.698019 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:49:26.697999    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01\": container with ID starting with c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01 not found: ID does not exist" containerID="c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"
Sep 09 13:49:26.698077 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.698027    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01"} err="failed to get container status \"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01\": rpc error: code = NotFound desc = could not find container \"c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01\": container with ID starting with c3f34e933e87de1319df094a431bea427cfec8c5f87c732a808523f4424bec01 not found: ID does not exist"
Sep 09 13:49:26.710837 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.710772    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:49:26.715274 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:26.715250    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-8f2df2/sub-test-model-8f2df2-kserve-797698b9cf-62489"]
Sep 09 13:49:27.618752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:49:27.618722    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" path="/var/lib/kubelet/pods/bdc4e5cf-f0e9-4dea-b62d-066db153141a/volumes"
Sep 09 13:50:00.556734 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.556702    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:50:00.557189 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.556920    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" podUID="c41ba6f5-4549-425a-a705-5f09318e34c5" containerName="maas-api" containerID="cri-o://8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d" gracePeriod=30
Sep 09 13:50:00.825301 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.825246    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:50:00.905718 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.905672    2559 generic.go:356] "Generic (PLEG): container finished" podID="c41ba6f5-4549-425a-a705-5f09318e34c5" containerID="8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d" exitCode=0
Sep 09 13:50:00.905881 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.905733    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"
Sep 09 13:50:00.905881 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.905761    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" event={"ID":"c41ba6f5-4549-425a-a705-5f09318e34c5","Type":"ContainerDied","Data":"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"}
Sep 09 13:50:00.905881 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.905834    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t" event={"ID":"c41ba6f5-4549-425a-a705-5f09318e34c5","Type":"ContainerDied","Data":"02675148acab5e1c0328c34c786343abb8e71503c0d16f273b04705e5a64937a"}
Sep 09 13:50:00.905881 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.905868    2559 scope.go:117] "RemoveContainer" containerID="8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"
Sep 09 13:50:00.915054 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.915036    2559 scope.go:117] "RemoveContainer" containerID="8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"
Sep 09 13:50:00.915374 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:50:00.915353    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d\": container with ID starting with 8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d not found: ID does not exist" containerID="8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"
Sep 09 13:50:00.915437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.915381    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d"} err="failed to get container status \"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d\": rpc error: code = NotFound desc = could not find container \"8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d\": container with ID starting with 8c1550d7be9ce13f363510194ab8c8965ea35134f4cb47bac81335b40d16f90d not found: ID does not exist"
Sep 09 13:50:00.938018 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.937993    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls\") pod \"c41ba6f5-4549-425a-a705-5f09318e34c5\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") "
Sep 09 13:50:00.938132 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.938111    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs\") pod \"c41ba6f5-4549-425a-a705-5f09318e34c5\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") "
Sep 09 13:50:00.938196 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.938172    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca\") pod \"c41ba6f5-4549-425a-a705-5f09318e34c5\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") "
Sep 09 13:50:00.938256 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.938220    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r6g5\" (UniqueName: \"kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5\") pod \"c41ba6f5-4549-425a-a705-5f09318e34c5\" (UID: \"c41ba6f5-4549-425a-a705-5f09318e34c5\") "
Sep 09 13:50:00.938614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.938547    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c41ba6f5-4549-425a-a705-5f09318e34c5" (UID: "c41ba6f5-4549-425a-a705-5f09318e34c5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:50:00.940615 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.940371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c41ba6f5-4549-425a-a705-5f09318e34c5" (UID: "c41ba6f5-4549-425a-a705-5f09318e34c5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:00.941057 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.941033    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c41ba6f5-4549-425a-a705-5f09318e34c5" (UID: "c41ba6f5-4549-425a-a705-5f09318e34c5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:00.941241 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:00.941219    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5" (OuterVolumeSpecName: "kube-api-access-6r6g5") pod "c41ba6f5-4549-425a-a705-5f09318e34c5" (UID: "c41ba6f5-4549-425a-a705-5f09318e34c5"). InnerVolumeSpecName "kube-api-access-6r6g5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:50:01.040053 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.040022    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:01.040053 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.040049    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41ba6f5-4549-425a-a705-5f09318e34c5-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:01.040053 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.040060    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41ba6f5-4549-425a-a705-5f09318e34c5-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:01.040278 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.040071    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r6g5\" (UniqueName: \"kubernetes.io/projected/c41ba6f5-4549-425a-a705-5f09318e34c5-kube-api-access-6r6g5\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:01.235590 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.235550    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:50:01.243061 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.243032    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-464c54-b7db4dc58-d2j6t"]
Sep 09 13:50:01.620144 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:01.620065    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41ba6f5-4549-425a-a705-5f09318e34c5" path="/var/lib/kubelet/pods/c41ba6f5-4549-425a-a705-5f09318e34c5/volumes"
Sep 09 13:50:23.313273 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.313238    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:50:23.313789 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.313460    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" podUID="0e9dd4bc-a101-4400-90c0-b018cb17b637" containerName="maas-api" containerID="cri-o://3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb" gracePeriod=30
Sep 09 13:50:23.561020 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.560999    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:50:23.654967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.654897    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs\") pod \"0e9dd4bc-a101-4400-90c0-b018cb17b637\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") "
Sep 09 13:50:23.654967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.654939    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfh78\" (UniqueName: \"kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78\") pod \"0e9dd4bc-a101-4400-90c0-b018cb17b637\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") "
Sep 09 13:50:23.655191 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.654981    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") pod \"0e9dd4bc-a101-4400-90c0-b018cb17b637\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") "
Sep 09 13:50:23.655191 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.655002    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca\") pod \"0e9dd4bc-a101-4400-90c0-b018cb17b637\" (UID: \"0e9dd4bc-a101-4400-90c0-b018cb17b637\") "
Sep 09 13:50:23.655437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.655406    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0e9dd4bc-a101-4400-90c0-b018cb17b637" (UID: "0e9dd4bc-a101-4400-90c0-b018cb17b637"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:50:23.657045 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.657015    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0e9dd4bc-a101-4400-90c0-b018cb17b637" (UID: "0e9dd4bc-a101-4400-90c0-b018cb17b637"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:23.657154 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.657045    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0e9dd4bc-a101-4400-90c0-b018cb17b637" (UID: "0e9dd4bc-a101-4400-90c0-b018cb17b637"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:23.657154 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.657117    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78" (OuterVolumeSpecName: "kube-api-access-bfh78") pod "0e9dd4bc-a101-4400-90c0-b018cb17b637" (UID: "0e9dd4bc-a101-4400-90c0-b018cb17b637"). InnerVolumeSpecName "kube-api-access-bfh78". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:50:23.755690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.755650    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:23.755690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.755683    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0e9dd4bc-a101-4400-90c0-b018cb17b637-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:23.755690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.755697    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0e9dd4bc-a101-4400-90c0-b018cb17b637-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:23.755940 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:23.755710    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfh78\" (UniqueName: \"kubernetes.io/projected/0e9dd4bc-a101-4400-90c0-b018cb17b637-kube-api-access-bfh78\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:24.053903 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.053864    2559 generic.go:356] "Generic (PLEG): container finished" podID="0e9dd4bc-a101-4400-90c0-b018cb17b637" containerID="3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb" exitCode=0
Sep 09 13:50:24.054073 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.053911    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"
Sep 09 13:50:24.054073 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.053937    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" event={"ID":"0e9dd4bc-a101-4400-90c0-b018cb17b637","Type":"ContainerDied","Data":"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"}
Sep 09 13:50:24.054073 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.053991    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6" event={"ID":"0e9dd4bc-a101-4400-90c0-b018cb17b637","Type":"ContainerDied","Data":"a54603f6450d72af05511a8e4de6002522d775458f32cab785ab2808e8bd9b25"}
Sep 09 13:50:24.054073 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.054020    2559 scope.go:117] "RemoveContainer" containerID="3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"
Sep 09 13:50:24.063683 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.063601    2559 scope.go:117] "RemoveContainer" containerID="3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"
Sep 09 13:50:24.063884 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:50:24.063863    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb\": container with ID starting with 3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb not found: ID does not exist" containerID="3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"
Sep 09 13:50:24.063958 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.063897    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb"} err="failed to get container status \"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb\": rpc error: code = NotFound desc = could not find container \"3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb\": container with ID starting with 3890b011716449a31324c517fb51cfda5c3f7b664e57150a7237d1c44afbf8bb not found: ID does not exist"
Sep 09 13:50:24.086758 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.086735    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:50:24.095690 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:24.095667    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-4bfe1d-599f5fd59-hj2l6"]
Sep 09 13:50:25.618429 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:25.618398    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e9dd4bc-a101-4400-90c0-b018cb17b637" path="/var/lib/kubelet/pods/0e9dd4bc-a101-4400-90c0-b018cb17b637/volumes"
Sep 09 13:50:41.346627 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.346591    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:50:41.347171 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.346862    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" podUID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" containerName="maas-api" containerID="cri-o://edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0" gracePeriod=30
Sep 09 13:50:41.609533 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.609474    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:50:41.725168 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725133    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs\") pod \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") "
Sep 09 13:50:41.725352 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725314    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xhnw\" (UniqueName: \"kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw\") pod \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") "
Sep 09 13:50:41.725425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725353    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca\") pod \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") "
Sep 09 13:50:41.725425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725378    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls\") pod \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\" (UID: \"b5cf250f-b749-48df-a7e5-f6a42cbb1774\") "
Sep 09 13:50:41.725738 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725701    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5cf250f-b749-48df-a7e5-f6a42cbb1774" (UID: "b5cf250f-b749-48df-a7e5-f6a42cbb1774"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:50:41.725874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.725829    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5cf250f-b749-48df-a7e5-f6a42cbb1774-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:41.727435 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.727411    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5cf250f-b749-48df-a7e5-f6a42cbb1774" (UID: "b5cf250f-b749-48df-a7e5-f6a42cbb1774"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:41.727735 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.727711    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b5cf250f-b749-48df-a7e5-f6a42cbb1774" (UID: "b5cf250f-b749-48df-a7e5-f6a42cbb1774"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:41.727735 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.727722    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw" (OuterVolumeSpecName: "kube-api-access-8xhnw") pod "b5cf250f-b749-48df-a7e5-f6a42cbb1774" (UID: "b5cf250f-b749-48df-a7e5-f6a42cbb1774"). InnerVolumeSpecName "kube-api-access-8xhnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:50:41.826684 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.826650    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xhnw\" (UniqueName: \"kubernetes.io/projected/b5cf250f-b749-48df-a7e5-f6a42cbb1774-kube-api-access-8xhnw\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:41.826684 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.826681    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:41.826879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:41.826695    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5cf250f-b749-48df-a7e5-f6a42cbb1774-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:42.164344 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.164308    2559 generic.go:356] "Generic (PLEG): container finished" podID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" containerID="edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0" exitCode=0
Sep 09 13:50:42.164524 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.164368    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"
Sep 09 13:50:42.164524 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.164358    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" event={"ID":"b5cf250f-b749-48df-a7e5-f6a42cbb1774","Type":"ContainerDied","Data":"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"}
Sep 09 13:50:42.164524 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.164501    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh" event={"ID":"b5cf250f-b749-48df-a7e5-f6a42cbb1774","Type":"ContainerDied","Data":"5cea9578e8628af4b90012ebc3b7d0426eadf28e79914a39c6ff4ee2cfb28d96"}
Sep 09 13:50:42.164741 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.164533    2559 scope.go:117] "RemoveContainer" containerID="edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"
Sep 09 13:50:42.176425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.176403    2559 scope.go:117] "RemoveContainer" containerID="edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"
Sep 09 13:50:42.176867 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:50:42.176846    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0\": container with ID starting with edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0 not found: ID does not exist" containerID="edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"
Sep 09 13:50:42.176971 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.176873    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0"} err="failed to get container status \"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0\": rpc error: code = NotFound desc = could not find container \"edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0\": container with ID starting with edf4a5c0a42c1c0225115ee1751b55463ca9e785ca3be7d23cf2a9e2855187b0 not found: ID does not exist"
Sep 09 13:50:42.193918 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.193897    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:50:42.198466 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:42.198444    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b5cba4-7674bbc77d-lgvvh"]
Sep 09 13:50:43.617454 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:43.617414    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" path="/var/lib/kubelet/pods/b5cf250f-b749-48df-a7e5-f6a42cbb1774/volumes"
Sep 09 13:50:58.926316 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:58.926280    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:50:58.926871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:58.926536    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" podUID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" containerName="maas-api" containerID="cri-o://fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd" gracePeriod=30
Sep 09 13:50:59.185450 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.185421    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:50:59.268611 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.268565    2559 generic.go:356] "Generic (PLEG): container finished" podID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" containerID="fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd" exitCode=0
Sep 09 13:50:59.268772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.268623    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"
Sep 09 13:50:59.268772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.268663    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" event={"ID":"e13b2ea3-eb7c-431c-9d17-7f315f25dd33","Type":"ContainerDied","Data":"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"}
Sep 09 13:50:59.268772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.268721    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m" event={"ID":"e13b2ea3-eb7c-431c-9d17-7f315f25dd33","Type":"ContainerDied","Data":"b7d879e1f152e6e60f26407d3abcbbd0d9710b00d5eaa46b3cb20138758a64c1"}
Sep 09 13:50:59.268772 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.268751    2559 scope.go:117] "RemoveContainer" containerID="fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"
Sep 09 13:50:59.276808 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.276787    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs\") pod \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") "
Sep 09 13:50:59.276953 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.276922    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") pod \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") "
Sep 09 13:50:59.277025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.276982    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca\") pod \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") "
Sep 09 13:50:59.277025 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.277011    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c4lkj\" (UniqueName: \"kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj\") pod \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\" (UID: \"e13b2ea3-eb7c-431c-9d17-7f315f25dd33\") "
Sep 09 13:50:59.277399 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.277368    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e13b2ea3-eb7c-431c-9d17-7f315f25dd33" (UID: "e13b2ea3-eb7c-431c-9d17-7f315f25dd33"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:50:59.278056 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.278033    2559 scope.go:117] "RemoveContainer" containerID="fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"
Sep 09 13:50:59.278366 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:50:59.278340    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd\": container with ID starting with fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd not found: ID does not exist" containerID="fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"
Sep 09 13:50:59.278432 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.278379    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd"} err="failed to get container status \"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd\": rpc error: code = NotFound desc = could not find container \"fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd\": container with ID starting with fd5d5241fb8a91c8bdad88e67841a583b2d86fdd51a0effa8d036e42f6be17cd not found: ID does not exist"
Sep 09 13:50:59.279148 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.279130    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj" (OuterVolumeSpecName: "kube-api-access-c4lkj") pod "e13b2ea3-eb7c-431c-9d17-7f315f25dd33" (UID: "e13b2ea3-eb7c-431c-9d17-7f315f25dd33"). InnerVolumeSpecName "kube-api-access-c4lkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:50:59.279326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.279311    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e13b2ea3-eb7c-431c-9d17-7f315f25dd33" (UID: "e13b2ea3-eb7c-431c-9d17-7f315f25dd33"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:59.279376 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.279339    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e13b2ea3-eb7c-431c-9d17-7f315f25dd33" (UID: "e13b2ea3-eb7c-431c-9d17-7f315f25dd33"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:50:59.377936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.377902    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:59.377936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.377929    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:59.377936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.377940    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4lkj\" (UniqueName: \"kubernetes.io/projected/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-kube-api-access-c4lkj\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:59.378168 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.377948    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e13b2ea3-eb7c-431c-9d17-7f315f25dd33-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:50:59.594124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.593471    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:50:59.604176 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.604142    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-6a7dbc-588569b5c7-cmx9m"]
Sep 09 13:50:59.618929 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:50:59.618892    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" path="/var/lib/kubelet/pods/e13b2ea3-eb7c-431c-9d17-7f315f25dd33/volumes"
Sep 09 13:51:12.701024 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.700990    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:51:12.701702 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701686    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" containerName="maas-api"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701705    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" containerName="maas-api"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701721    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41ba6f5-4549-425a-a705-5f09318e34c5" containerName="maas-api"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701729    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41ba6f5-4549-425a-a705-5f09318e34c5" containerName="maas-api"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701746    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" containerName="main"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701755    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" containerName="main"
Sep 09 13:51:12.701765 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701763    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e9dd4bc-a101-4400-90c0-b018cb17b637" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701772    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e9dd4bc-a101-4400-90c0-b018cb17b637" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701791    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701799    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701904    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5cf250f-b749-48df-a7e5-f6a42cbb1774" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701915    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdc4e5cf-f0e9-4dea-b62d-066db153141a" containerName="main"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701926    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41ba6f5-4549-425a-a705-5f09318e34c5" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701934    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e9dd4bc-a101-4400-90c0-b018cb17b637" containerName="maas-api"
Sep 09 13:51:12.701955 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.701943    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13b2ea3-eb7c-431c-9d17-7f315f25dd33" containerName="maas-api"
Sep 09 13:51:12.706968 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.706941    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.709697 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.709676    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-8349bf-serving-cert\""
Sep 09 13:51:12.717987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.717964    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:51:12.808656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.808616    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.808824 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.808774    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.808871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.808825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.808871 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.808853    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td8hr\" (UniqueName: \"kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.909497 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.909467    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.909725 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.909542    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.909725 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.909593    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.909725 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.909616    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td8hr\" (UniqueName: \"kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.910304 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.910275    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.912007 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.911982    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.912124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.911985    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:12.918305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:12.918278    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td8hr\" (UniqueName: \"kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr\") pod \"maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:13.024325 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:13.024298    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:13.311230 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:13.311193    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:51:13.314294 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:51:13.314260    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b77da7c_a332_4652_83d6_33411d083a8c.slice/crio-701e641c585e0306bf245509223c82961adf092723e7d5a119e84d492c39f65c WatchSource:0}: Error finding container 701e641c585e0306bf245509223c82961adf092723e7d5a119e84d492c39f65c: Status 404 returned error can't find the container with id 701e641c585e0306bf245509223c82961adf092723e7d5a119e84d492c39f65c
Sep 09 13:51:13.358723 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:13.358687    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" event={"ID":"6b77da7c-a332-4652-83d6-33411d083a8c","Type":"ContainerStarted","Data":"701e641c585e0306bf245509223c82961adf092723e7d5a119e84d492c39f65c"}
Sep 09 13:51:15.373418 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:15.373369    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" event={"ID":"6b77da7c-a332-4652-83d6-33411d083a8c","Type":"ContainerStarted","Data":"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"}
Sep 09 13:51:15.373896 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:15.373547    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:15.394122 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:15.394071    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" podStartSLOduration=2.392004837 podStartE2EDuration="3.394058806s" totalImagesPullingTime="1.002053969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:51:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:51:15.391496466 +0000 UTC m=+1494.280196458" watchObservedRunningTime="2026-09-09 13:51:15.394058806 +0000 UTC m=+1494.282758796"
Sep 09 13:51:21.385378 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:21.385338    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:51:42.160374 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.160334    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:51:42.164172 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.164148    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.166479 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.166455    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-3691cf-serving-cert\""
Sep 09 13:51:42.166812 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.166790    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.166891 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.166825    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mshg\" (UniqueName: \"kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.166987 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.166950    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.167110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.167092    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.175858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.175835    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:51:42.268585 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.268534    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.268887 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.268594    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.268887 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.268618    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mshg\" (UniqueName: \"kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.268887 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.268683    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.268887 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:51:42.268719    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-serving-cert: secret "maas-api-e2e-ipp-b-3691cf-serving-cert" not found
Sep 09 13:51:42.268887 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:51:42.268862    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls podName:071d306b-b934-4414-a1ca-a70387249c6f nodeName:}" failed. No retries permitted until 2026-09-09 13:51:42.768771686 +0000 UTC m=+1521.657471673 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls") pod "maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" (UID: "071d306b-b934-4414-a1ca-a70387249c6f") : secret "maas-api-e2e-ipp-b-3691cf-serving-cert" not found
Sep 09 13:51:42.269389 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.269352    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.271685 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.271656    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.279435 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.279412    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mshg\" (UniqueName: \"kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.774423 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.774386    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.776975 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.776946    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") pod \"maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:42.779743 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:42.779717    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:43.062165 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:43.062137    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:51:43.063294 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:51:43.063266    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod071d306b_b934_4414_a1ca_a70387249c6f.slice/crio-5a7564d6fb76fd2e0a72d47b434f9645422ea7f69231a2d238a9893b1983c750 WatchSource:0}: Error finding container 5a7564d6fb76fd2e0a72d47b434f9645422ea7f69231a2d238a9893b1983c750: Status 404 returned error can't find the container with id 5a7564d6fb76fd2e0a72d47b434f9645422ea7f69231a2d238a9893b1983c750
Sep 09 13:51:43.539133 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:43.539099    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" event={"ID":"071d306b-b934-4414-a1ca-a70387249c6f","Type":"ContainerStarted","Data":"5a7564d6fb76fd2e0a72d47b434f9645422ea7f69231a2d238a9893b1983c750"}
Sep 09 13:51:44.546347 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:44.546307    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" event={"ID":"071d306b-b934-4414-a1ca-a70387249c6f","Type":"ContainerStarted","Data":"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"}
Sep 09 13:51:44.546784 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:44.546388    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:51:44.564102 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:44.564061    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" podStartSLOduration=1.473372201 podStartE2EDuration="2.564049828s" totalImagesPullingTime="1.090677627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:51:44.563817 +0000 UTC m=+1523.452516989" watchObservedRunningTime="2026-09-09 13:51:44.564049828 +0000 UTC m=+1523.452749818"
Sep 09 13:51:50.558331 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:51:50.558299    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:52:10.805279 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:52:10.805247    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 13:53:37.895902 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:37.895823    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 13:53:42.499247 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.499210    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:53:42.504271 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.504226    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.507349 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.507321    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-b2555b-serving-cert\""
Sep 09 13:53:42.513911 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.513259    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:53:42.591219 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.591166    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.591391 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.591348    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7f5b\" (UniqueName: \"kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.591439 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.591399    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.591476 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.591454    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.693020 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.692980    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7f5b\" (UniqueName: \"kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.693210 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.693026    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.693210 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.693068    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.693210 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.693124    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.693210 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:53:42.693164    2559 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-serving-cert: secret "maas-api-e2e-ipp-cleanup-b2555b-serving-cert" not found
Sep 09 13:53:42.693374 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:53:42.693235    2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls podName:d71d7804-4878-4b81-9af5-c53505eeaaa6 nodeName:}" failed. No retries permitted until 2026-09-09 13:53:43.193214413 +0000 UTC m=+1642.081914388 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" (UID: "d71d7804-4878-4b81-9af5-c53505eeaaa6") : secret "maas-api-e2e-ipp-cleanup-b2555b-serving-cert" not found
Sep 09 13:53:42.693962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.693932    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.695764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.695733    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:42.701643 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:42.701619    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7f5b\" (UniqueName: \"kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:43.197746 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:43.197708    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:43.200408 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:43.200378    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:43.422748 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:43.422705    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:43.713218 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:43.713171    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:53:43.717290 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:53:43.717257    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd71d7804_4878_4b81_9af5_c53505eeaaa6.slice/crio-3061e5b1724843fdc1d3d261162a031e4f4ebeee797bd19f58c8b61958c7190f WatchSource:0}: Error finding container 3061e5b1724843fdc1d3d261162a031e4f4ebeee797bd19f58c8b61958c7190f: Status 404 returned error can't find the container with id 3061e5b1724843fdc1d3d261162a031e4f4ebeee797bd19f58c8b61958c7190f
Sep 09 13:53:44.282766 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:44.282726    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" event={"ID":"d71d7804-4878-4b81-9af5-c53505eeaaa6","Type":"ContainerStarted","Data":"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"}
Sep 09 13:53:44.282967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:44.282780    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" event={"ID":"d71d7804-4878-4b81-9af5-c53505eeaaa6","Type":"ContainerStarted","Data":"3061e5b1724843fdc1d3d261162a031e4f4ebeee797bd19f58c8b61958c7190f"}
Sep 09 13:53:44.282967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:44.282817    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:53:44.305005 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:44.304943    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" podStartSLOduration=1.844797234 podStartE2EDuration="2.304922798s" totalImagesPullingTime="460.125564ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:53:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:53:44.303375653 +0000 UTC m=+1643.192075645" watchObservedRunningTime="2026-09-09 13:53:44.304922798 +0000 UTC m=+1643.193622788"
Sep 09 13:53:50.294145 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:53:50.294111    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:54:13.090852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.090815    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:54:13.091381 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.091051    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" podUID="d71d7804-4878-4b81-9af5-c53505eeaaa6" containerName="maas-api" containerID="cri-o://b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9" gracePeriod=30
Sep 09 13:54:13.345767 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.345704    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:54:13.472137 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.472096    2559 generic.go:356] "Generic (PLEG): container finished" podID="d71d7804-4878-4b81-9af5-c53505eeaaa6" containerID="b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9" exitCode=0
Sep 09 13:54:13.472305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.472159    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"
Sep 09 13:54:13.472305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.472183    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" event={"ID":"d71d7804-4878-4b81-9af5-c53505eeaaa6","Type":"ContainerDied","Data":"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"}
Sep 09 13:54:13.472305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.472266    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4" event={"ID":"d71d7804-4878-4b81-9af5-c53505eeaaa6","Type":"ContainerDied","Data":"3061e5b1724843fdc1d3d261162a031e4f4ebeee797bd19f58c8b61958c7190f"}
Sep 09 13:54:13.472305 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.472290    2559 scope.go:117] "RemoveContainer" containerID="b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"
Sep 09 13:54:13.482281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482196    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7f5b\" (UniqueName: \"kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b\") pod \"d71d7804-4878-4b81-9af5-c53505eeaaa6\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") "
Sep 09 13:54:13.482281 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482255    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs\") pod \"d71d7804-4878-4b81-9af5-c53505eeaaa6\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") "
Sep 09 13:54:13.482420 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482290    2559 scope.go:117] "RemoveContainer" containerID="b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"
Sep 09 13:54:13.482420 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482319    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca\") pod \"d71d7804-4878-4b81-9af5-c53505eeaaa6\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") "
Sep 09 13:54:13.482420 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482340    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") pod \"d71d7804-4878-4b81-9af5-c53505eeaaa6\" (UID: \"d71d7804-4878-4b81-9af5-c53505eeaaa6\") "
Sep 09 13:54:13.482709 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:54:13.482664    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9\": container with ID starting with b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9 not found: ID does not exist" containerID="b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"
Sep 09 13:54:13.482709 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482699    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9"} err="failed to get container status \"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9\": rpc error: code = NotFound desc = could not find container \"b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9\": container with ID starting with b673a40a5163313266336eb256f0debf5d7eb3539d6de84ea931a7ee8b6148c9 not found: ID does not exist"
Sep 09 13:54:13.482844 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.482697    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d71d7804-4878-4b81-9af5-c53505eeaaa6" (UID: "d71d7804-4878-4b81-9af5-c53505eeaaa6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:54:13.484599 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.484553    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d71d7804-4878-4b81-9af5-c53505eeaaa6" (UID: "d71d7804-4878-4b81-9af5-c53505eeaaa6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:54:13.484852 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.484820    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d71d7804-4878-4b81-9af5-c53505eeaaa6" (UID: "d71d7804-4878-4b81-9af5-c53505eeaaa6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:54:13.484943 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.484874    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b" (OuterVolumeSpecName: "kube-api-access-m7f5b") pod "d71d7804-4878-4b81-9af5-c53505eeaaa6" (UID: "d71d7804-4878-4b81-9af5-c53505eeaaa6"). InnerVolumeSpecName "kube-api-access-m7f5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:54:13.584080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.584049    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7f5b\" (UniqueName: \"kubernetes.io/projected/d71d7804-4878-4b81-9af5-c53505eeaaa6-kube-api-access-m7f5b\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:13.584080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.584078    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:13.584080 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.584087    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d71d7804-4878-4b81-9af5-c53505eeaaa6-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:13.584284 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.584095    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d71d7804-4878-4b81-9af5-c53505eeaaa6-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:13.795634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.795605    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:54:13.799933 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:13.799910    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-b2555b-d7d98d57f-fz8k4"]
Sep 09 13:54:15.617803 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:15.617769    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d71d7804-4878-4b81-9af5-c53505eeaaa6" path="/var/lib/kubelet/pods/d71d7804-4878-4b81-9af5-c53505eeaaa6/volumes"
Sep 09 13:54:38.703808 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:38.703772    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:54:38.704600 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:38.704551    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" podUID="6b77da7c-a332-4652-83d6-33411d083a8c" containerName="maas-api" containerID="cri-o://dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a" gracePeriod=30
Sep 09 13:54:38.970740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:38.970715    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:54:39.014992 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.014960    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca\") pod \"6b77da7c-a332-4652-83d6-33411d083a8c\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") "
Sep 09 13:54:39.015160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.015003    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-td8hr\" (UniqueName: \"kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr\") pod \"6b77da7c-a332-4652-83d6-33411d083a8c\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") "
Sep 09 13:54:39.015160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.015055    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs\") pod \"6b77da7c-a332-4652-83d6-33411d083a8c\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") "
Sep 09 13:54:39.015160 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.015091    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls\") pod \"6b77da7c-a332-4652-83d6-33411d083a8c\" (UID: \"6b77da7c-a332-4652-83d6-33411d083a8c\") "
Sep 09 13:54:39.015393 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.015362    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b77da7c-a332-4652-83d6-33411d083a8c" (UID: "6b77da7c-a332-4652-83d6-33411d083a8c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:54:39.017359 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.017328    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b77da7c-a332-4652-83d6-33411d083a8c" (UID: "6b77da7c-a332-4652-83d6-33411d083a8c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:54:39.017504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.017386    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr" (OuterVolumeSpecName: "kube-api-access-td8hr") pod "6b77da7c-a332-4652-83d6-33411d083a8c" (UID: "6b77da7c-a332-4652-83d6-33411d083a8c"). InnerVolumeSpecName "kube-api-access-td8hr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:54:39.017504 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.017413    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b77da7c-a332-4652-83d6-33411d083a8c" (UID: "6b77da7c-a332-4652-83d6-33411d083a8c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:54:39.115967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.115936    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:39.115967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.115962    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b77da7c-a332-4652-83d6-33411d083a8c-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:39.115967 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.115971    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b77da7c-a332-4652-83d6-33411d083a8c-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:39.116266 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.115981    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-td8hr\" (UniqueName: \"kubernetes.io/projected/6b77da7c-a332-4652-83d6-33411d083a8c-kube-api-access-td8hr\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:54:39.630764 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.630729    2559 generic.go:356] "Generic (PLEG): container finished" podID="6b77da7c-a332-4652-83d6-33411d083a8c" containerID="dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a" exitCode=0
Sep 09 13:54:39.630874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.630798    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"
Sep 09 13:54:39.630874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.630804    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" event={"ID":"6b77da7c-a332-4652-83d6-33411d083a8c","Type":"ContainerDied","Data":"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"}
Sep 09 13:54:39.630874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.630861    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9" event={"ID":"6b77da7c-a332-4652-83d6-33411d083a8c","Type":"ContainerDied","Data":"701e641c585e0306bf245509223c82961adf092723e7d5a119e84d492c39f65c"}
Sep 09 13:54:39.630990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.630883    2559 scope.go:117] "RemoveContainer" containerID="dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"
Sep 09 13:54:39.639870 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.639852    2559 scope.go:117] "RemoveContainer" containerID="dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"
Sep 09 13:54:39.640134 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:54:39.640115    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a\": container with ID starting with dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a not found: ID does not exist" containerID="dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"
Sep 09 13:54:39.640201 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.640147    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a"} err="failed to get container status \"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a\": rpc error: code = NotFound desc = could not find container \"dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a\": container with ID starting with dca9fd6953bb6dd7900ec628806d3c839eb7b9cc6c3029e9bd69b7583dfd587a not found: ID does not exist"
Sep 09 13:54:39.649899 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.649843    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:54:39.654658 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:39.654637    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-8349bf-78f9b79ddf-pp6b9"]
Sep 09 13:54:41.618014 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:54:41.617984    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b77da7c-a332-4652-83d6-33411d083a8c" path="/var/lib/kubelet/pods/6b77da7c-a332-4652-83d6-33411d083a8c/volumes"
Sep 09 13:55:38.286887 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.286826    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:55:38.289425 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.287201    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" podUID="071d306b-b934-4414-a1ca-a70387249c6f" containerName="maas-api" containerID="cri-o://7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd" gracePeriod=30
Sep 09 13:55:38.547848 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.547788    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:55:38.641029 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.640997    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs\") pod \"071d306b-b934-4414-a1ca-a70387249c6f\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") "
Sep 09 13:55:38.641229 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.641037    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca\") pod \"071d306b-b934-4414-a1ca-a70387249c6f\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") "
Sep 09 13:55:38.641229 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.641113    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") pod \"071d306b-b934-4414-a1ca-a70387249c6f\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") "
Sep 09 13:55:38.641229 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.641188    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mshg\" (UniqueName: \"kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg\") pod \"071d306b-b934-4414-a1ca-a70387249c6f\" (UID: \"071d306b-b934-4414-a1ca-a70387249c6f\") "
Sep 09 13:55:38.641990 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.641954    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "071d306b-b934-4414-a1ca-a70387249c6f" (UID: "071d306b-b934-4414-a1ca-a70387249c6f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:55:38.643841 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.643809    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "071d306b-b934-4414-a1ca-a70387249c6f" (UID: "071d306b-b934-4414-a1ca-a70387249c6f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:55:38.644004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.643985    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg" (OuterVolumeSpecName: "kube-api-access-4mshg") pod "071d306b-b934-4414-a1ca-a70387249c6f" (UID: "071d306b-b934-4414-a1ca-a70387249c6f"). InnerVolumeSpecName "kube-api-access-4mshg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:55:38.644070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.644014    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "071d306b-b934-4414-a1ca-a70387249c6f" (UID: "071d306b-b934-4414-a1ca-a70387249c6f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:55:38.742106 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.742075    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:55:38.742106 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.742103    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mshg\" (UniqueName: \"kubernetes.io/projected/071d306b-b934-4414-a1ca-a70387249c6f-kube-api-access-4mshg\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:55:38.742106 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.742112    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/071d306b-b934-4414-a1ca-a70387249c6f-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:55:38.742326 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.742121    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/071d306b-b934-4414-a1ca-a70387249c6f-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:55:38.977711 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.977676    2559 generic.go:356] "Generic (PLEG): container finished" podID="071d306b-b934-4414-a1ca-a70387249c6f" containerID="7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd" exitCode=0
Sep 09 13:55:38.977879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.977723    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"
Sep 09 13:55:38.977879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.977731    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" event={"ID":"071d306b-b934-4414-a1ca-a70387249c6f","Type":"ContainerDied","Data":"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"}
Sep 09 13:55:38.977879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.977785    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw" event={"ID":"071d306b-b934-4414-a1ca-a70387249c6f","Type":"ContainerDied","Data":"5a7564d6fb76fd2e0a72d47b434f9645422ea7f69231a2d238a9893b1983c750"}
Sep 09 13:55:38.977879 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.977817    2559 scope.go:117] "RemoveContainer" containerID="7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"
Sep 09 13:55:38.994969 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.994947    2559 scope.go:117] "RemoveContainer" containerID="7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"
Sep 09 13:55:38.995231 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:55:38.995214    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd\": container with ID starting with 7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd not found: ID does not exist" containerID="7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"
Sep 09 13:55:38.995287 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:38.995245    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd"} err="failed to get container status \"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd\": rpc error: code = NotFound desc = could not find container \"7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd\": container with ID starting with 7ef92cbd5534c462f087636d6daf7f8f61ac34e6ef859aced2476e0f554270dd not found: ID does not exist"
Sep 09 13:55:39.005506 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:39.005480    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:55:39.011149 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:39.011123    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-3691cf-658f9b59bd-p8hfw"]
Sep 09 13:55:39.618141 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:55:39.618112    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="071d306b-b934-4414-a1ca-a70387249c6f" path="/var/lib/kubelet/pods/071d306b-b934-4414-a1ca-a70387249c6f/volumes"
Sep 09 13:56:13.260872 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.260838    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:56:13.261446 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.261101    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" podUID="99cbbb71-5088-479b-856c-536f7af2bd9f" containerName="maas-api" containerID="cri-o://8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a" gracePeriod=30
Sep 09 13:56:13.314538 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:56:13.314499    2559 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cbbb71_5088_479b_856c_536f7af2bd9f.slice/crio-conmon-8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99cbbb71_5088_479b_856c_536f7af2bd9f.slice/crio-8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a.scope\": RecentStats: unable to find data in memory cache]"
Sep 09 13:56:13.520359 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.520300    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:56:13.559777 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.559746    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs\") pod \"99cbbb71-5088-479b-856c-536f7af2bd9f\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") "
Sep 09 13:56:13.559936 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.559823    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca\") pod \"99cbbb71-5088-479b-856c-536f7af2bd9f\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") "
Sep 09 13:56:13.560301 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.560271    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") pod \"99cbbb71-5088-479b-856c-536f7af2bd9f\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") "
Sep 09 13:56:13.560445 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.560346    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm98t\" (UniqueName: \"kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t\") pod \"99cbbb71-5088-479b-856c-536f7af2bd9f\" (UID: \"99cbbb71-5088-479b-856c-536f7af2bd9f\") "
Sep 09 13:56:13.560634 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.560601    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "99cbbb71-5088-479b-856c-536f7af2bd9f" (UID: "99cbbb71-5088-479b-856c-536f7af2bd9f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:56:13.563150 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.563127    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/99cbbb71-5088-479b-856c-536f7af2bd9f-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:13.564612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.563371    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "99cbbb71-5088-479b-856c-536f7af2bd9f" (UID: "99cbbb71-5088-479b-856c-536f7af2bd9f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:56:13.570744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.570715    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "99cbbb71-5088-479b-856c-536f7af2bd9f" (UID: "99cbbb71-5088-479b-856c-536f7af2bd9f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:56:13.570744 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.570732    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t" (OuterVolumeSpecName: "kube-api-access-dm98t") pod "99cbbb71-5088-479b-856c-536f7af2bd9f" (UID: "99cbbb71-5088-479b-856c-536f7af2bd9f"). InnerVolumeSpecName "kube-api-access-dm98t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:56:13.664945 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.664914    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:13.665124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.664950    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm98t\" (UniqueName: \"kubernetes.io/projected/99cbbb71-5088-479b-856c-536f7af2bd9f-kube-api-access-dm98t\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:13.665124 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:13.664966    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/99cbbb71-5088-479b-856c-536f7af2bd9f-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:14.187414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.187332    2559 generic.go:356] "Generic (PLEG): container finished" podID="99cbbb71-5088-479b-856c-536f7af2bd9f" containerID="8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a" exitCode=0
Sep 09 13:56:14.187414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.187397    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"
Sep 09 13:56:14.187612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.187419    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" event={"ID":"99cbbb71-5088-479b-856c-536f7af2bd9f","Type":"ContainerDied","Data":"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"}
Sep 09 13:56:14.187612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.187483    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt" event={"ID":"99cbbb71-5088-479b-856c-536f7af2bd9f","Type":"ContainerDied","Data":"7633418c434fc7bf545bca62cea6c9ecb404a45b7352b548b77db2a6fb2b8e19"}
Sep 09 13:56:14.187612 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.187511    2559 scope.go:117] "RemoveContainer" containerID="8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"
Sep 09 13:56:14.196639 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.196621    2559 scope.go:117] "RemoveContainer" containerID="8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"
Sep 09 13:56:14.196908 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:56:14.196890    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a\": container with ID starting with 8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a not found: ID does not exist" containerID="8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"
Sep 09 13:56:14.196964 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.196916    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a"} err="failed to get container status \"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a\": rpc error: code = NotFound desc = could not find container \"8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a\": container with ID starting with 8adc154ce85453fb03b6242c615876f94841ae940df884ba5e6182959bccc85a not found: ID does not exist"
Sep 09 13:56:14.207307 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.207283    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:56:14.211491 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:14.211469    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-df48ae-78b569cb7c-zkxzt"]
Sep 09 13:56:15.617133 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:15.617091    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99cbbb71-5088-479b-856c-536f7af2bd9f" path="/var/lib/kubelet/pods/99cbbb71-5088-479b-856c-536f7af2bd9f/volumes"
Sep 09 13:56:22.893434 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:22.893396    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:56:22.893977 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:22.893697    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" podUID="171df6d2-84af-40ed-87de-befa44e517f5" containerName="manager" containerID="cri-o://68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3" gracePeriod=30
Sep 09 13:56:23.137387 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.137361    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:56:23.242231 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.242203    2559 generic.go:356] "Generic (PLEG): container finished" podID="171df6d2-84af-40ed-87de-befa44e517f5" containerID="68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3" exitCode=0
Sep 09 13:56:23.242414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.242240    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" event={"ID":"171df6d2-84af-40ed-87de-befa44e517f5","Type":"ContainerDied","Data":"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"}
Sep 09 13:56:23.242414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.242246    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-wmlcx"
Sep 09 13:56:23.242414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.242283    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-wmlcx" event={"ID":"171df6d2-84af-40ed-87de-befa44e517f5","Type":"ContainerDied","Data":"152d90b133f62b9624143833bd7a23ac26ed70fca1d5386a1adb51c30efe1cb1"}
Sep 09 13:56:23.242414 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.242307    2559 scope.go:117] "RemoveContainer" containerID="68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"
Sep 09 13:56:23.250740 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.250716    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert\") pod \"171df6d2-84af-40ed-87de-befa44e517f5\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") "
Sep 09 13:56:23.250874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.250753    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4f6tt\" (UniqueName: \"kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt\") pod \"171df6d2-84af-40ed-87de-befa44e517f5\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") "
Sep 09 13:56:23.250874 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.250783    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs\") pod \"171df6d2-84af-40ed-87de-befa44e517f5\" (UID: \"171df6d2-84af-40ed-87de-befa44e517f5\") "
Sep 09 13:56:23.251298 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.251213    2559 scope.go:117] "RemoveContainer" containerID="68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"
Sep 09 13:56:23.251562 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:56:23.251542    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3\": container with ID starting with 68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3 not found: ID does not exist" containerID="68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"
Sep 09 13:56:23.251656 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.251589    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3"} err="failed to get container status \"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3\": rpc error: code = NotFound desc = could not find container \"68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3\": container with ID starting with 68d97fb15299cfb8786fd485bcd288e9efb27e41c84d5577c1dc210a742e30d3 not found: ID does not exist"
Sep 09 13:56:23.253345 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.253318    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "171df6d2-84af-40ed-87de-befa44e517f5" (UID: "171df6d2-84af-40ed-87de-befa44e517f5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:56:23.253447 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.253343    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt" (OuterVolumeSpecName: "kube-api-access-4f6tt") pod "171df6d2-84af-40ed-87de-befa44e517f5" (UID: "171df6d2-84af-40ed-87de-befa44e517f5"). InnerVolumeSpecName "kube-api-access-4f6tt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:56:23.253447 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.253342    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "171df6d2-84af-40ed-87de-befa44e517f5" (UID: "171df6d2-84af-40ed-87de-befa44e517f5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:56:23.352302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.352278    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-webhook-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:23.352302 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.352299    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4f6tt\" (UniqueName: \"kubernetes.io/projected/171df6d2-84af-40ed-87de-befa44e517f5-kube-api-access-4f6tt\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:23.352427 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.352309    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/171df6d2-84af-40ed-87de-befa44e517f5-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:56:23.570161 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.570134    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:56:23.573743 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.573720    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-wmlcx"]
Sep 09 13:56:23.617920 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:56:23.617890    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="171df6d2-84af-40ed-87de-befa44e517f5" path="/var/lib/kubelet/pods/171df6d2-84af-40ed-87de-befa44e517f5/volumes"
Sep 09 13:57:24.401556 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.401515    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:24.402161 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402139    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="071d306b-b934-4414-a1ca-a70387249c6f" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402163    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="071d306b-b934-4414-a1ca-a70387249c6f" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402178    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99cbbb71-5088-479b-856c-536f7af2bd9f" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402186    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="99cbbb71-5088-479b-856c-536f7af2bd9f" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402196    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d71d7804-4878-4b81-9af5-c53505eeaaa6" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402205    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d71d7804-4878-4b81-9af5-c53505eeaaa6" containerName="maas-api"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402220    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="171df6d2-84af-40ed-87de-befa44e517f5" containerName="manager"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402227    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="171df6d2-84af-40ed-87de-befa44e517f5" containerName="manager"
Sep 09 13:57:24.402254 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402252    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b77da7c-a332-4652-83d6-33411d083a8c" containerName="maas-api"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402261    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b77da7c-a332-4652-83d6-33411d083a8c" containerName="maas-api"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402379    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b77da7c-a332-4652-83d6-33411d083a8c" containerName="maas-api"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402391    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d71d7804-4878-4b81-9af5-c53505eeaaa6" containerName="maas-api"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402401    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="171df6d2-84af-40ed-87de-befa44e517f5" containerName="manager"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402411    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="99cbbb71-5088-479b-856c-536f7af2bd9f" containerName="maas-api"
Sep 09 13:57:24.402693 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.402425    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="071d306b-b934-4414-a1ca-a70387249c6f" containerName="maas-api"
Sep 09 13:57:24.407034 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.407011    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.409851 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.409829    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:57:24.410989 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.410964    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 13:57:24.411096 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.410995    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 13:57:24.411096 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.411051    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8lp8w\""
Sep 09 13:57:24.417217 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.417195    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:24.464883 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.464860    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.465004 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.464909    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pf2n\" (UniqueName: \"kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.465047 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.465029    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.565961 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.565932    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.566070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.565972    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.566070 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.566003    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pf2n\" (UniqueName: \"kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.568374 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.568344    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.568467 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.568369    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.575835 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.575810    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pf2n\" (UniqueName: \"kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n\") pod \"maas-controller-59b6f545bd-bmtmq\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") " pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:24.718084 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:24.718057    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:25.002950 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:25.002923    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:25.005715 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:57:25.005687    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e620760_7aa6_48d7_9f86_cd2aadb049df.slice/crio-10e069eb8499741cdf045118f3a0f5ca286922a019175ee96a80743d6d8c090e WatchSource:0}: Error finding container 10e069eb8499741cdf045118f3a0f5ca286922a019175ee96a80743d6d8c090e: Status 404 returned error can't find the container with id 10e069eb8499741cdf045118f3a0f5ca286922a019175ee96a80743d6d8c090e
Sep 09 13:57:25.007060 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:25.007040    2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:57:25.619350 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:25.619215    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" event={"ID":"7e620760-7aa6-48d7-9f86-cd2aadb049df","Type":"ContainerStarted","Data":"10e069eb8499741cdf045118f3a0f5ca286922a019175ee96a80743d6d8c090e"}
Sep 09 13:57:26.622546 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:26.622508    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" event={"ID":"7e620760-7aa6-48d7-9f86-cd2aadb049df","Type":"ContainerStarted","Data":"dc05a5df5ba4e35321b077fd66481e5b65b589f0db234427bc52e19b6f1a035e"}
Sep 09 13:57:26.623016 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:26.622596    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:26.647252 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:26.647202    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" podStartSLOduration=2.106908438 podStartE2EDuration="2.647188312s" totalImagesPullingTime="540.279874ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:57:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:57:26.644464317 +0000 UTC m=+1865.533164308" watchObservedRunningTime="2026-09-09 13:57:26.647188312 +0000 UTC m=+1865.535888321"
Sep 09 13:57:37.630698 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:37.630666    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:47.014283 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.014179    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:47.014760 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.014594    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerName="manager" containerID="cri-o://dc05a5df5ba4e35321b077fd66481e5b65b589f0db234427bc52e19b6f1a035e" gracePeriod=30
Sep 09 13:57:47.629447 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.629283    2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.91:8081/readyz\": dial tcp 10.133.0.91:8081: connect: connection refused"
Sep 09 13:57:47.748506 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.748477    2559 generic.go:356] "Generic (PLEG): container finished" podID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerID="dc05a5df5ba4e35321b077fd66481e5b65b589f0db234427bc52e19b6f1a035e" exitCode=0
Sep 09 13:57:47.748686 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.748529    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" event={"ID":"7e620760-7aa6-48d7-9f86-cd2aadb049df","Type":"ContainerDied","Data":"dc05a5df5ba4e35321b077fd66481e5b65b589f0db234427bc52e19b6f1a035e"}
Sep 09 13:57:47.768851 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.768828    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:47.863982 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.863940    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs\") pod \"7e620760-7aa6-48d7-9f86-cd2aadb049df\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") "
Sep 09 13:57:47.864153 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.864043    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pf2n\" (UniqueName: \"kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n\") pod \"7e620760-7aa6-48d7-9f86-cd2aadb049df\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") "
Sep 09 13:57:47.864153 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.864088    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert\") pod \"7e620760-7aa6-48d7-9f86-cd2aadb049df\" (UID: \"7e620760-7aa6-48d7-9f86-cd2aadb049df\") "
Sep 09 13:57:47.866016 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.865980    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e620760-7aa6-48d7-9f86-cd2aadb049df" (UID: "7e620760-7aa6-48d7-9f86-cd2aadb049df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:57:47.866142 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.866058    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n" (OuterVolumeSpecName: "kube-api-access-7pf2n") pod "7e620760-7aa6-48d7-9f86-cd2aadb049df" (UID: "7e620760-7aa6-48d7-9f86-cd2aadb049df"). InnerVolumeSpecName "kube-api-access-7pf2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:57:47.866198 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.866140    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7e620760-7aa6-48d7-9f86-cd2aadb049df" (UID: "7e620760-7aa6-48d7-9f86-cd2aadb049df"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:57:47.965411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.965384    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pf2n\" (UniqueName: \"kubernetes.io/projected/7e620760-7aa6-48d7-9f86-cd2aadb049df-kube-api-access-7pf2n\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:57:47.965411 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.965412    2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-webhook-cert\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:57:47.965560 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:47.965421    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e620760-7aa6-48d7-9f86-cd2aadb049df-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:57:48.755376 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:48.755350    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-bmtmq"
Sep 09 13:57:48.755825 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:48.755384    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-bmtmq" event={"ID":"7e620760-7aa6-48d7-9f86-cd2aadb049df","Type":"ContainerDied","Data":"10e069eb8499741cdf045118f3a0f5ca286922a019175ee96a80743d6d8c090e"}
Sep 09 13:57:48.755825 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:48.755465    2559 scope.go:117] "RemoveContainer" containerID="dc05a5df5ba4e35321b077fd66481e5b65b589f0db234427bc52e19b6f1a035e"
Sep 09 13:57:48.784057 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:48.784030    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:48.788250 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:48.788222    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-bmtmq"]
Sep 09 13:57:49.593368 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.593329    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-59b6f545bd-n8t82"]
Sep 09 13:57:49.593839 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.593823    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerName="manager"
Sep 09 13:57:49.593884 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.593841    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerName="manager"
Sep 09 13:57:49.593923 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.593899    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" containerName="manager"
Sep 09 13:57:49.598094 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.598077    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.601699 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.601679    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 13:57:49.602770 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.602750    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:57:49.602770 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.602766    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8lp8w\""
Sep 09 13:57:49.602897 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.602751    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 13:57:49.621047 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.621018    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e620760-7aa6-48d7-9f86-cd2aadb049df" path="/var/lib/kubelet/pods/7e620760-7aa6-48d7-9f86-cd2aadb049df/volumes"
Sep 09 13:57:49.621357 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.621338    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-n8t82"]
Sep 09 13:57:49.678850 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.678815    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-webhook-cert\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.679538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.679215    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-metrics-certs\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.679538 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.679322    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2j65\" (UniqueName: \"kubernetes.io/projected/ebae7f71-801a-4fb0-855a-de2282cf64e8-kube-api-access-t2j65\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.780860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.780829    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-metrics-certs\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.780860 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.780861    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2j65\" (UniqueName: \"kubernetes.io/projected/ebae7f71-801a-4fb0-855a-de2282cf64e8-kube-api-access-t2j65\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.781258 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.780918    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-webhook-cert\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.783367 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.783342    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-webhook-cert\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.783473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.783411    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ebae7f71-801a-4fb0-855a-de2282cf64e8-metrics-certs\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.790472 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.790448    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2j65\" (UniqueName: \"kubernetes.io/projected/ebae7f71-801a-4fb0-855a-de2282cf64e8-kube-api-access-t2j65\") pod \"maas-controller-59b6f545bd-n8t82\" (UID: \"ebae7f71-801a-4fb0-855a-de2282cf64e8\") " pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:49.908496 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:49.908423    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:50.185212 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:50.185183    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-59b6f545bd-n8t82"]
Sep 09 13:57:50.185749 ip-10-0-136-35 kubenswrapper[2559]: W0909 13:57:50.185725    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebae7f71_801a_4fb0_855a_de2282cf64e8.slice/crio-7939da0a3fe16882664bb450b79e30b300ceec8e9f8fe20625cca21723b58a3b WatchSource:0}: Error finding container 7939da0a3fe16882664bb450b79e30b300ceec8e9f8fe20625cca21723b58a3b: Status 404 returned error can't find the container with id 7939da0a3fe16882664bb450b79e30b300ceec8e9f8fe20625cca21723b58a3b
Sep 09 13:57:50.769437 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:50.769396    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-n8t82" event={"ID":"ebae7f71-801a-4fb0-855a-de2282cf64e8","Type":"ContainerStarted","Data":"cea0e5b0ef4a98dcd42dafb0990dcb7575c1869f89684688d67b42b4dcd0b8a8"}
Sep 09 13:57:50.769614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:50.769462    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-59b6f545bd-n8t82" event={"ID":"ebae7f71-801a-4fb0-855a-de2282cf64e8","Type":"ContainerStarted","Data":"7939da0a3fe16882664bb450b79e30b300ceec8e9f8fe20625cca21723b58a3b"}
Sep 09 13:57:50.769614 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:50.769567    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:57:50.800962 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:57:50.800872    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-59b6f545bd-n8t82" podStartSLOduration=1.4400033190000001 podStartE2EDuration="1.800857573s" totalImagesPullingTime="360.854254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:57:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:57:50.7995072 +0000 UTC m=+1889.688207192" watchObservedRunningTime="2026-09-09 13:57:50.800857573 +0000 UTC m=+1889.689557563"
Sep 09 13:58:01.776785 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:58:01.776747    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-59b6f545bd-n8t82"
Sep 09 13:59:18.329035 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:18.328950    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:59:18.329473 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:18.329200    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" podUID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" containerName="manager" containerID="cri-o://7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401" gracePeriod=10
Sep 09 13:59:19.184715 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.184688    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:59:19.263752 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.263709    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume\") pod \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") "
Sep 09 13:59:19.263965 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.263774    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82sj5\" (UniqueName: \"kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5\") pod \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\" (UID: \"e44bd4fa-16b0-4040-a1d7-4109a3ebce97\") "
Sep 09 13:59:19.264110 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.264082    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e44bd4fa-16b0-4040-a1d7-4109a3ebce97" (UID: "e44bd4fa-16b0-4040-a1d7-4109a3ebce97"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:59:19.265816 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.265790    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5" (OuterVolumeSpecName: "kube-api-access-82sj5") pod "e44bd4fa-16b0-4040-a1d7-4109a3ebce97" (UID: "e44bd4fa-16b0-4040-a1d7-4109a3ebce97"). InnerVolumeSpecName "kube-api-access-82sj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:59:19.289709 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.289674    2559 generic.go:356] "Generic (PLEG): container finished" podID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" containerID="7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401" exitCode=0
Sep 09 13:59:19.289858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.289720    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"
Sep 09 13:59:19.289858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.289764    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" event={"ID":"e44bd4fa-16b0-4040-a1d7-4109a3ebce97","Type":"ContainerDied","Data":"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"}
Sep 09 13:59:19.289858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.289825    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp" event={"ID":"e44bd4fa-16b0-4040-a1d7-4109a3ebce97","Type":"ContainerDied","Data":"ecb655fe574ac11710c0d35c6a448f89882a3c24a2ec84271e1535bffcc21757"}
Sep 09 13:59:19.289858 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.289849    2559 scope.go:117] "RemoveContainer" containerID="7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"
Sep 09 13:59:19.299557 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.299531    2559 scope.go:117] "RemoveContainer" containerID="7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"
Sep 09 13:59:19.299862 ip-10-0-136-35 kubenswrapper[2559]: E0909 13:59:19.299846    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401\": container with ID starting with 7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401 not found: ID does not exist" containerID="7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"
Sep 09 13:59:19.299947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.299871    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401"} err="failed to get container status \"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401\": rpc error: code = NotFound desc = could not find container \"7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401\": container with ID starting with 7e74cce7f191c10a0d16e26dd6bbfd04e2761ab872bbc7a4ea212a967577d401 not found: ID does not exist"
Sep 09 13:59:19.364947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.364902    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82sj5\" (UniqueName: \"kubernetes.io/projected/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-kube-api-access-82sj5\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:59:19.364947 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.364932    2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e44bd4fa-16b0-4040-a1d7-4109a3ebce97-extensions-socket-volume\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 13:59:19.379925 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.379884    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:59:19.397395 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.397355    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-ltbtp"]
Sep 09 13:59:19.618854 ip-10-0-136-35 kubenswrapper[2559]: I0909 13:59:19.618774    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" path="/var/lib/kubelet/pods/e44bd4fa-16b0-4040-a1d7-4109a3ebce97/volumes"
Sep 09 14:00:19.076850 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.076808    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"]
Sep 09 14:00:19.077286 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.077245    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" containerName="manager"
Sep 09 14:00:19.077286 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.077257    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" containerName="manager"
Sep 09 14:00:19.077359 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.077323    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e44bd4fa-16b0-4040-a1d7-4109a3ebce97" containerName="manager"
Sep 09 14:00:19.080525 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.080505    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.084895 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.084863    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-g67j8\""
Sep 09 14:00:19.121472 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.121436    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"]
Sep 09 14:00:19.181400 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.181363    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zp5rp\" (UniqueName: \"kubernetes.io/projected/daf83458-5156-4de7-b389-a30e2dc58395-kube-api-access-zp5rp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.181637 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.181463    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daf83458-5156-4de7-b389-a30e2dc58395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.282538 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.282500    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daf83458-5156-4de7-b389-a30e2dc58395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.282777 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.282646    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zp5rp\" (UniqueName: \"kubernetes.io/projected/daf83458-5156-4de7-b389-a30e2dc58395-kube-api-access-zp5rp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.282970 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.282944    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/daf83458-5156-4de7-b389-a30e2dc58395-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.294991 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.294958    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zp5rp\" (UniqueName: \"kubernetes.io/projected/daf83458-5156-4de7-b389-a30e2dc58395-kube-api-access-zp5rp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-lflss\" (UID: \"daf83458-5156-4de7-b389-a30e2dc58395\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.391820 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.391718    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:19.773915 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:19.773881    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"]
Sep 09 14:00:19.775224 ip-10-0-136-35 kubenswrapper[2559]: W0909 14:00:19.775176    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaf83458_5156_4de7_b389_a30e2dc58395.slice/crio-8c223fa9a2e58666ebd19c30b50d18d9f1684c24127a3ac1f6aaeb340f77784d WatchSource:0}: Error finding container 8c223fa9a2e58666ebd19c30b50d18d9f1684c24127a3ac1f6aaeb340f77784d: Status 404 returned error can't find the container with id 8c223fa9a2e58666ebd19c30b50d18d9f1684c24127a3ac1f6aaeb340f77784d
Sep 09 14:00:20.653180 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:20.653133    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss" event={"ID":"daf83458-5156-4de7-b389-a30e2dc58395","Type":"ContainerStarted","Data":"3e6d2fcb3a5d2f613f28cd1e24b2ef30b6be652e261e17b3a43b4ab710983044"}
Sep 09 14:00:20.653665 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:20.653196    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss" event={"ID":"daf83458-5156-4de7-b389-a30e2dc58395","Type":"ContainerStarted","Data":"8c223fa9a2e58666ebd19c30b50d18d9f1684c24127a3ac1f6aaeb340f77784d"}
Sep 09 14:00:20.653665 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:20.653307    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:00:20.728375 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:20.728309    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss" podStartSLOduration=1.728292613 podStartE2EDuration="1.728292613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 14:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 14:00:20.726740523 +0000 UTC m=+2039.615440513" watchObservedRunningTime="2026-09-09 14:00:20.728292613 +0000 UTC m=+2039.616992603"
Sep 09 14:00:31.661905 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:00:31.661869    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-lflss"
Sep 09 14:02:14.896255 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:14.896173    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"]
Sep 09 14:02:14.900295 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:14.900275    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:14.909940 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:14.909912    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"]
Sep 09 14:02:15.033380 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.033331    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtsbd\" (UniqueName: \"kubernetes.io/projected/7036350c-9d40-4475-860f-8e48c6ce0f9d-kube-api-access-rtsbd\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.033567 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.033396    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-maas-api-tls\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.033567 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.033455    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7036350c-9d40-4475-860f-8e48c6ce0f9d-openshift-service-ca\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.033702 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.033558    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-metrics-certs\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.135054 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.135007    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7036350c-9d40-4475-860f-8e48c6ce0f9d-openshift-service-ca\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.135264 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.135098    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-metrics-certs\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.135264 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.135240    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtsbd\" (UniqueName: \"kubernetes.io/projected/7036350c-9d40-4475-860f-8e48c6ce0f9d-kube-api-access-rtsbd\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.135669 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.135643    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-maas-api-tls\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.135811 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.135789    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7036350c-9d40-4475-860f-8e48c6ce0f9d-openshift-service-ca\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.137816 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.137789    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-metrics-certs\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.138543 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.138523    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7036350c-9d40-4475-860f-8e48c6ce0f9d-maas-api-tls\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.149613 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.149532    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtsbd\" (UniqueName: \"kubernetes.io/projected/7036350c-9d40-4475-860f-8e48c6ce0f9d-kube-api-access-rtsbd\") pod \"maas-api-8585bfb869-kjvpv\" (UID: \"7036350c-9d40-4475-860f-8e48c6ce0f9d\") " pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.215445 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.215404    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:15.500539 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:15.500507    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"]
Sep 09 14:02:15.500961 ip-10-0-136-35 kubenswrapper[2559]: W0909 14:02:15.500932    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7036350c_9d40_4475_860f_8e48c6ce0f9d.slice/crio-1aa7da9f98b81c5407fa185065c84d4baedbf8cce18396b24c5c78cff05185c4 WatchSource:0}: Error finding container 1aa7da9f98b81c5407fa185065c84d4baedbf8cce18396b24c5c78cff05185c4: Status 404 returned error can't find the container with id 1aa7da9f98b81c5407fa185065c84d4baedbf8cce18396b24c5c78cff05185c4
Sep 09 14:02:16.351975 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:16.351938    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv" event={"ID":"7036350c-9d40-4475-860f-8e48c6ce0f9d","Type":"ContainerStarted","Data":"888427d26738d6a40da352acff90fb01d0e8c5bf882167010687283aed26903c"}
Sep 09 14:02:16.352335 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:16.352002    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv" event={"ID":"7036350c-9d40-4475-860f-8e48c6ce0f9d","Type":"ContainerStarted","Data":"1aa7da9f98b81c5407fa185065c84d4baedbf8cce18396b24c5c78cff05185c4"}
Sep 09 14:02:16.352335 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:16.352166    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:16.373018 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:16.372968    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv" podStartSLOduration=2.031126234 podStartE2EDuration="2.372953935s" totalImagesPullingTime="341.827701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 14:02:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 14:02:16.369119616 +0000 UTC m=+2155.257819606" watchObservedRunningTime="2026-09-09 14:02:16.372953935 +0000 UTC m=+2155.261653946"
Sep 09 14:02:22.363386 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:22.363351    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8585bfb869-kjvpv"
Sep 09 14:02:22.418781 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:22.418749    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 14:02:22.418992 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:22.418949    2559 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" podUID="940f8b83-9a42-4d17-9c06-39ac71fb9547" containerName="maas-api" containerID="cri-o://f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7" gracePeriod=30
Sep 09 14:02:23.672762 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.672740    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 14:02:23.819095 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.819012    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l886q\" (UniqueName: \"kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q\") pod \"940f8b83-9a42-4d17-9c06-39ac71fb9547\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") "
Sep 09 14:02:23.819237 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.819129    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs\") pod \"940f8b83-9a42-4d17-9c06-39ac71fb9547\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") "
Sep 09 14:02:23.819237 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.819157    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") pod \"940f8b83-9a42-4d17-9c06-39ac71fb9547\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") "
Sep 09 14:02:23.819237 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.819191    2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca\") pod \"940f8b83-9a42-4d17-9c06-39ac71fb9547\" (UID: \"940f8b83-9a42-4d17-9c06-39ac71fb9547\") "
Sep 09 14:02:23.819635 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.819600    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "940f8b83-9a42-4d17-9c06-39ac71fb9547" (UID: "940f8b83-9a42-4d17-9c06-39ac71fb9547"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 14:02:23.821191 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.821159    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "940f8b83-9a42-4d17-9c06-39ac71fb9547" (UID: "940f8b83-9a42-4d17-9c06-39ac71fb9547"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 14:02:23.821191 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.821176    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q" (OuterVolumeSpecName: "kube-api-access-l886q") pod "940f8b83-9a42-4d17-9c06-39ac71fb9547" (UID: "940f8b83-9a42-4d17-9c06-39ac71fb9547"). InnerVolumeSpecName "kube-api-access-l886q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 14:02:23.821324 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.821231    2559 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "940f8b83-9a42-4d17-9c06-39ac71fb9547" (UID: "940f8b83-9a42-4d17-9c06-39ac71fb9547"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 14:02:23.920925 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.920885    2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l886q\" (UniqueName: \"kubernetes.io/projected/940f8b83-9a42-4d17-9c06-39ac71fb9547-kube-api-access-l886q\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 14:02:23.920925 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.920920    2559 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-metrics-certs\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 14:02:23.920925 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.920930    2559 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/940f8b83-9a42-4d17-9c06-39ac71fb9547-maas-api-tls\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 14:02:23.921151 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:23.920938    2559 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/940f8b83-9a42-4d17-9c06-39ac71fb9547-openshift-service-ca\") on node \"ip-10-0-136-35.ec2.internal\" DevicePath \"\""
Sep 09 14:02:24.406496 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.406460    2559 generic.go:356] "Generic (PLEG): container finished" podID="940f8b83-9a42-4d17-9c06-39ac71fb9547" containerID="f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7" exitCode=0
Sep 09 14:02:24.406704 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.406510    2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"
Sep 09 14:02:24.406704 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.406518    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" event={"ID":"940f8b83-9a42-4d17-9c06-39ac71fb9547","Type":"ContainerDied","Data":"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"}
Sep 09 14:02:24.406704 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.406645    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x" event={"ID":"940f8b83-9a42-4d17-9c06-39ac71fb9547","Type":"ContainerDied","Data":"19bfc6c7be997eec547cc1a969980b92aa813c5f088f6e24c517170598d756a5"}
Sep 09 14:02:24.406704 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.406682    2559 scope.go:117] "RemoveContainer" containerID="f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"
Sep 09 14:02:24.416858 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.416841    2559 scope.go:117] "RemoveContainer" containerID="f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"
Sep 09 14:02:24.417138 ip-10-0-136-35 kubenswrapper[2559]: E0909 14:02:24.417122    2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7\": container with ID starting with f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7 not found: ID does not exist" containerID="f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"
Sep 09 14:02:24.417181 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.417143    2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7"} err="failed to get container status \"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7\": rpc error: code = NotFound desc = could not find container \"f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7\": container with ID starting with f73523ea73a15d3db8b5186e217119825ba2cfd9576a82aa035e870edfea3eb7 not found: ID does not exist"
Sep 09 14:02:24.429683 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.429660    2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 14:02:24.437936 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:24.437915    2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6b4b89c5b5-dhs6x"]
Sep 09 14:02:25.616948 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:25.616908    2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="940f8b83-9a42-4d17-9c06-39ac71fb9547" path="/var/lib/kubelet/pods/940f8b83-9a42-4d17-9c06-39ac71fb9547/volumes"
Sep 09 14:02:36.598607 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:36.598564    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64889fc9d6-fqr9m_7f4eb108-effa-4ba9-a918-96ab305ff003/authorino/0.log"
Sep 09 14:02:41.860039 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:41.860008    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6cpsw_9ed2da18-6939-455b-81c5-c05a0ed8e50f/manager/0.log"
Sep 09 14:02:41.978085 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:41.978055    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q448v_7f26a138-c968-44fc-afe7-64480e465ba3/manager/0.log"
Sep 09 14:02:42.102011 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:42.101984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59b6f545bd-n8t82_ebae7f71-801a-4fb0-855a-de2282cf64e8/manager/0.log"
Sep 09 14:02:42.245332 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:42.245308    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4644v_f43a4e83-5ca1-4670-bdcd-3daee12c5c17/server/0.log"
Sep 09 14:02:42.363317 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:42.363292    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pslf5_a8ecf428-7915-466c-810d-9cfc32a081d7/manager/0.log"
Sep 09 14:02:42.810178 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:42.810149    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8469c9486d-pjp87_38df386b-935d-46ec-a35c-c213c0d9db09/copy-manifests/0.log"
Sep 09 14:02:42.818164 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:42.818146    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8469c9486d-pjp87_38df386b-935d-46ec-a35c-c213c0d9db09/manager/0.log"
Sep 09 14:02:43.260039 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:43.260016    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8585bfb869-kjvpv_7036350c-9d40-4475-860f-8e48c6ce0f9d/maas-api/0.log"
Sep 09 14:02:43.588485 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:43.588396    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-crtxx_2db5bf17-2f8b-4d3d-a2b2-5c4131db1b12/postgres/0.log"
Sep 09 14:02:44.368612 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:44.368562    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64889fc9d6-fqr9m_7f4eb108-effa-4ba9-a918-96ab305ff003/authorino/0.log"
Sep 09 14:02:44.485176 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:44.485144    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-76fd5_d97d04ee-24b0-4ed1-975d-54c05dd219a9/manager/0.log"
Sep 09 14:02:44.596781 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:44.596752    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-qprnj_ec21a6e4-4aa8-46bc-9540-36b3b2d993a9/manager/0.log"
Sep 09 14:02:44.709533 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:44.709505    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-fs2bh_dd9dac10-198a-4aba-9909-e056c7e61546/kuadrant-console-plugin/0.log"
Sep 09 14:02:44.828540 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:44.828513    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-lflss_daf83458-5156-4de7-b389-a30e2dc58395/manager/0.log"
Sep 09 14:02:45.066934 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:45.066853    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-pqqzl_22dc8db1-1151-42d3-b76a-4846747bcd9b/manager/0.log"
Sep 09 14:02:45.638488 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:45.638459    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-595c6d67d9-p8vtt_6735ab87-3b5f-485d-8923-2e47396086ee/kube-auth-proxy/0.log"
Sep 09 14:02:45.858676 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:45.858645    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/1.log"
Sep 09 14:02:45.967970 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:45.967942    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-mz5t5_d7375e01-5993-4511-b8bd-62c73387e3b1/payload-pre-processing/0.log"
Sep 09 14:02:46.083511 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:46.083485    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 14:02:46.775512 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:46.775478    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-v2ht6_96bcc051-9818-4e81-a1a1-7216424dbb0c/main/0.log"
Sep 09 14:02:55.434863 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:55.434829    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gv79z_e7315ff5-e3eb-421b-b3cd-f0460cb94ee4/global-pull-secret-syncer/0.log"
Sep 09 14:02:55.652365 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:55.652334    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xzxtz_912c2218-a6f4-40ff-b5e4-d7e06ce737e2/konnectivity-agent/0.log"
Sep 09 14:02:55.744814 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:02:55.744790    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-35.ec2.internal_5067553cfb83a070b3ee7a935b901cd8/haproxy/0.log"
Sep 09 14:03:00.350622 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.350587    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-64889fc9d6-fqr9m_7f4eb108-effa-4ba9-a918-96ab305ff003/authorino/0.log"
Sep 09 14:03:00.387067 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.387033    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-76fd5_d97d04ee-24b0-4ed1-975d-54c05dd219a9/manager/0.log"
Sep 09 14:03:00.417668 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.417632    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-qprnj_ec21a6e4-4aa8-46bc-9540-36b3b2d993a9/manager/0.log"
Sep 09 14:03:00.446985 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.446958    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-fs2bh_dd9dac10-198a-4aba-9909-e056c7e61546/kuadrant-console-plugin/0.log"
Sep 09 14:03:00.483884 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.483863    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-lflss_daf83458-5156-4de7-b389-a30e2dc58395/manager/0.log"
Sep 09 14:03:00.562503 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:00.562479    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-pqqzl_22dc8db1-1151-42d3-b76a-4846747bcd9b/manager/0.log"
Sep 09 14:03:02.077710 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.077677    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/alertmanager/0.log"
Sep 09 14:03:02.112672 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.112648    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/config-reloader/0.log"
Sep 09 14:03:02.140918 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.140891    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/kube-rbac-proxy-web/0.log"
Sep 09 14:03:02.167120 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.167099    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/kube-rbac-proxy/0.log"
Sep 09 14:03:02.195497 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.195478    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/kube-rbac-proxy-metric/0.log"
Sep 09 14:03:02.223873 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.223854    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/prom-label-proxy/0.log"
Sep 09 14:03:02.253776 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.253757    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_03b6c040-99cd-4055-90d5-cdf8620234d8/init-config-reloader/0.log"
Sep 09 14:03:02.337080 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.337026    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xxjbf_85df1f1d-b27e-42b0-987b-30db495b9939/kube-state-metrics/0.log"
Sep 09 14:03:02.357251 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.357229    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xxjbf_85df1f1d-b27e-42b0-987b-30db495b9939/kube-rbac-proxy-main/0.log"
Sep 09 14:03:02.380168 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.380143    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-xxjbf_85df1f1d-b27e-42b0-987b-30db495b9939/kube-rbac-proxy-self/0.log"
Sep 09 14:03:02.411837 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.411818    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-66648567fc-842m5_1a833e65-61dd-42c8-9303-a3db7e2b3d3f/metrics-server/0.log"
Sep 09 14:03:02.441119 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.441102    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-c4bbt_9edf72c6-c3ec-469e-926f-2adfcfd9b0a0/monitoring-plugin/0.log"
Sep 09 14:03:02.557152 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.557131    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz9g9_f929a2f6-b4fa-4d8e-86c5-20abd3764133/node-exporter/0.log"
Sep 09 14:03:02.585567 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.585548    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz9g9_f929a2f6-b4fa-4d8e-86c5-20abd3764133/kube-rbac-proxy/0.log"
Sep 09 14:03:02.612972 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:02.612924    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wz9g9_f929a2f6-b4fa-4d8e-86c5-20abd3764133/init-textfile/0.log"
Sep 09 14:03:03.072587 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:03.072546    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-98rb6_fd50e27d-7078-4e17-b6fd-9dd2352afba0/prometheus-operator/0.log"
Sep 09 14:03:03.111100 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:03.111073    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-6f4946db56-98rb6_fd50e27d-7078-4e17-b6fd-9dd2352afba0/kube-rbac-proxy/0.log"
Sep 09 14:03:03.190567 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:03.190505    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d58bc97cc-wpnt4_eb4d44ff-a5f9-4367-b151-3616768e602c/telemeter-client/0.log"
Sep 09 14:03:03.246313 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:03.246294    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d58bc97cc-wpnt4_eb4d44ff-a5f9-4367-b151-3616768e602c/reload/0.log"
Sep 09 14:03:03.277968 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:03.277951    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5d58bc97cc-wpnt4_eb4d44ff-a5f9-4367-b151-3616768e602c/kube-rbac-proxy/0.log"
Sep 09 14:03:06.065498 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.065467    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c97b554d8-7mbzm_bda15c46-7d5d-4999-b924-8964ac7f5d2e/console/0.log"
Sep 09 14:03:06.142001 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.141964    2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"]
Sep 09 14:03:06.142419 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.142402    2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="940f8b83-9a42-4d17-9c06-39ac71fb9547" containerName="maas-api"
Sep 09 14:03:06.142499 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.142421    2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="940f8b83-9a42-4d17-9c06-39ac71fb9547" containerName="maas-api"
Sep 09 14:03:06.142537 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.142508    2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="940f8b83-9a42-4d17-9c06-39ac71fb9547" containerName="maas-api"
Sep 09 14:03:06.145986 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.145966    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.150218 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.150196    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqbm9\"/\"openshift-service-ca.crt\""
Sep 09 14:03:06.150314 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.150233    2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqbm9\"/\"kube-root-ca.crt\""
Sep 09 14:03:06.150314 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.150203    2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqbm9\"/\"default-dockercfg-gttcx\""
Sep 09 14:03:06.155982 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.155963    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"]
Sep 09 14:03:06.278010 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.277975    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-lib-modules\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.278206 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.278026    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-podres\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.278206 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.278046    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqbfp\" (UniqueName: \"kubernetes.io/projected/aafa2882-a4ec-4e4f-b50d-9cc413baddba-kube-api-access-sqbfp\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.278206 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.278153    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-sys\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.278340 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.278208    2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-proc\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379044 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.378967    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-sys\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379044 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379018    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-proc\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379221 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379081    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-lib-modules\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379221 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379100    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-sys\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379221 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379131    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-podres\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379221 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379161    2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqbfp\" (UniqueName: \"kubernetes.io/projected/aafa2882-a4ec-4e4f-b50d-9cc413baddba-kube-api-access-sqbfp\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379221 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379165    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-proc\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379399 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379236    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-lib-modules\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.379399 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.379260    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/aafa2882-a4ec-4e4f-b50d-9cc413baddba-podres\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.406397 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.406375    2559 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqbfp\" (UniqueName: \"kubernetes.io/projected/aafa2882-a4ec-4e4f-b50d-9cc413baddba-kube-api-access-sqbfp\") pod \"perf-node-gather-daemonset-d6c4l\" (UID: \"aafa2882-a4ec-4e4f-b50d-9cc413baddba\") " pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.456750 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.456726    2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:06.730965 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:06.730940    2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"]
Sep 09 14:03:06.733079 ip-10-0-136-35 kubenswrapper[2559]: W0909 14:03:06.733056    2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podaafa2882_a4ec_4e4f_b50d_9cc413baddba.slice/crio-6fd634152089574323a19224961cf4b2dfffc0001669645e0573595936afd5cc WatchSource:0}: Error finding container 6fd634152089574323a19224961cf4b2dfffc0001669645e0573595936afd5cc: Status 404 returned error can't find the container with id 6fd634152089574323a19224961cf4b2dfffc0001669645e0573595936afd5cc
Sep 09 14:03:07.673229 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:07.673189    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l" event={"ID":"aafa2882-a4ec-4e4f-b50d-9cc413baddba","Type":"ContainerStarted","Data":"d141935ed7bf6463e4afc05494639d5a473df28badf2bf7b6ee92844a63f0fa9"}
Sep 09 14:03:07.673706 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:07.673258    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:07.673706 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:07.673287    2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l" event={"ID":"aafa2882-a4ec-4e4f-b50d-9cc413baddba","Type":"ContainerStarted","Data":"6fd634152089574323a19224961cf4b2dfffc0001669645e0573595936afd5cc"}
Sep 09 14:03:07.692042 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:07.691997    2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l" podStartSLOduration=1.691982842 podStartE2EDuration="1.691982842s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 14:03:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 14:03:07.690861682 +0000 UTC m=+2206.579561675" watchObservedRunningTime="2026-09-09 14:03:07.691982842 +0000 UTC m=+2206.580682832"
Sep 09 14:03:08.065651 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:08.065620    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kfr4_861fccec-5212-4e31-ba42-a620e1e8e4bb/dns/0.log"
Sep 09 14:03:08.091324 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:08.091301    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-7kfr4_861fccec-5212-4e31-ba42-a620e1e8e4bb/kube-rbac-proxy/0.log"
Sep 09 14:03:08.233712 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:08.233685    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-v2xgb_ffa1b527-a17f-4123-9ca0-33cf66176215/dns-node-resolver/0.log"
Sep 09 14:03:08.829508 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:08.829476    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6zfw7_c75d4892-66db-4119-bd92-5b343db8b44e/node-ca/0.log"
Sep 09 14:03:10.287835 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:10.287800    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-595c6d67d9-p8vtt_6735ab87-3b5f-485d-8923-2e47396086ee/kube-auth-proxy/0.log"
Sep 09 14:03:10.364868 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:10.364818    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/0.log"
Sep 09 14:03:10.374784 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:10.374752    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/1.log"
Sep 09 14:03:10.434795 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:10.434771    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-mz5t5_d7375e01-5993-4511-b8bd-62c73387e3b1/payload-pre-processing/0.log"
Sep 09 14:03:10.493972 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:10.493946    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 14:03:11.491787 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:11.491760    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-qh9sn_304f489b-4b99-4ff1-b5ee-469281fbdc59/serve-healthcheck-canary/0.log"
Sep 09 14:03:12.225563 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:12.225528    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zl4p7_f2626952-4ba1-4c76-a586-3764efb71eb1/kube-rbac-proxy/0.log"
Sep 09 14:03:12.252681 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:12.252658    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zl4p7_f2626952-4ba1-4c76-a586-3764efb71eb1/exporter/0.log"
Sep 09 14:03:12.279924 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:12.279899    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zl4p7_f2626952-4ba1-4c76-a586-3764efb71eb1/extractor/0.log"
Sep 09 14:03:13.689384 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:13.689357    2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqbm9/perf-node-gather-daemonset-d6c4l"
Sep 09 14:03:14.527288 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:14.527252    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-6cpsw_9ed2da18-6939-455b-81c5-c05a0ed8e50f/manager/0.log"
Sep 09 14:03:14.709212 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:14.709180    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-q448v_7f26a138-c968-44fc-afe7-64480e465ba3/manager/0.log"
Sep 09 14:03:14.757399 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:14.757368    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-59b6f545bd-n8t82_ebae7f71-801a-4fb0-855a-de2282cf64e8/manager/0.log"
Sep 09 14:03:14.787073 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:14.787021    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-4644v_f43a4e83-5ca1-4670-bdcd-3daee12c5c17/server/0.log"
Sep 09 14:03:14.869217 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:14.869195    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pslf5_a8ecf428-7915-466c-810d-9cfc32a081d7/manager/0.log"
Sep 09 14:03:15.062051 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:15.061984    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8469c9486d-pjp87_38df386b-935d-46ec-a35c-c213c0d9db09/manager/0.log"
Sep 09 14:03:15.086924 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:15.086901    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8469c9486d-pjp87_38df386b-935d-46ec-a35c-c213c0d9db09/copy-manifests/0.log"
Sep 09 14:03:16.491383 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:16.491359    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-595c6d67d9-p8vtt_6735ab87-3b5f-485d-8923-2e47396086ee/kube-auth-proxy/0.log"
Sep 09 14:03:16.529792 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:16.529761    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/0.log"
Sep 09 14:03:16.538019 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:16.538000    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/1.log"
Sep 09 14:03:16.563412 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:16.563390    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-mz5t5_d7375e01-5993-4511-b8bd-62c73387e3b1/payload-pre-processing/0.log"
Sep 09 14:03:16.594201 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:16.594181    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 14:03:18.564294 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:18.564251    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-595c6d67d9-p8vtt_6735ab87-3b5f-485d-8923-2e47396086ee/kube-auth-proxy/0.log"
Sep 09 14:03:18.611052 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:18.610959    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/0.log"
Sep 09 14:03:18.618688 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:18.618665    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-dldkt_411812a1-de71-41c8-8f62-e0e65a822c8b/istio-proxy/1.log"
Sep 09 14:03:18.645778 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:18.645755    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-mz5t5_d7375e01-5993-4511-b8bd-62c73387e3b1/payload-pre-processing/0.log"
Sep 09 14:03:18.672375 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:18.672346    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-4s7cw_38a9d046-9290-433e-b8ca-a75ab25e4abd/payload-processing/0.log"
Sep 09 14:03:21.787985 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:21.787953    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-p6kvg_bd179afe-17f5-4660-b662-b5299d388a83/migrator/0.log"
Sep 09 14:03:21.813827 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:21.813801    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-p6kvg_bd179afe-17f5-4660-b662-b5299d388a83/graceful-termination/0.log"
Sep 09 14:03:23.616285 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.616240    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/kube-multus-additional-cni-plugins/0.log"
Sep 09 14:03:23.643778 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.643751    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/egress-router-binary-copy/0.log"
Sep 09 14:03:23.667616 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.667590    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/cni-plugins/0.log"
Sep 09 14:03:23.693727 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.693697    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/bond-cni-plugin/0.log"
Sep 09 14:03:23.718282 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.718250    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/routeoverride-cni/0.log"
Sep 09 14:03:23.744385 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.744359    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/whereabouts-cni-bincopy/0.log"
Sep 09 14:03:23.769810 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.769781    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xrlmc_cfb90b51-1423-4e1f-8771-82dd73ab0bce/whereabouts-cni/0.log"
Sep 09 14:03:23.860619 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.860566    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-hk747_65bd199c-778e-45f1-a136-bc1ef27587ea/kube-multus/0.log"
Sep 09 14:03:23.942168 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.942069    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nfsvq_36eed273-3006-4797-978e-bf5d1f555385/network-metrics-daemon/0.log"
Sep 09 14:03:23.963436 ip-10-0-136-35 kubenswrapper[2559]: I0909 14:03:23.963389    2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nfsvq_36eed273-3006-4797-978e-bf5d1f555385/kube-rbac-proxy/0.log"