Sep 10 16:15:23.440927 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 10 16:15:23.441083 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 10 16:15:23.441198 ip-10-0-136-40 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 10 16:15:23.441747 ip-10-0-136-40 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 10 16:15:34.864642 ip-10-0-136-40 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 10 16:15:34.864660 ip-10-0-136-40 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 662a75d29d13489d86a9f976ce8c1c33 --
Sep 10 16:17:06.296581 ip-10-0-136-40 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 16:17:06.711957 ip-10-0-136-40 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:17:06.711957 ip-10-0-136-40 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 16:17:06.711957 ip-10-0-136-40 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:17:06.711957 ip-10-0-136-40 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 16:17:06.711957 ip-10-0-136-40 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 16:17:06.714613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.714525    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721460    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721479    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721484    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721487    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721490    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.721483 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721493    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721496    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721500    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721503    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721506    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721509    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721511    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721514    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721516    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721519    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721522    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721524    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721527    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721530    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721532    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721535    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721538    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721540    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721542    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721545    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.721709 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721548    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721550    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721553    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721555    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721558    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721560    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721564    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721566    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721569    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721572    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721574    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721577    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721579    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721582    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721585    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721588    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721590    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721593    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721596    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721599    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.722188 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721602    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721604    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721607    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721610    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721612    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721615    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721617    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721620    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721622    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721625    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721627    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721630    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721633    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721636    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721638    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721641    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721643    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721648    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721652    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.722700 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721655    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721660    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721663    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721666    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721669    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721672    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721675    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721679    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721682    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721685    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721688    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721690    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721693    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721696    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721698    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721701    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721704    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721707    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721709    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721713    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.723160 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721716    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721719    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721723    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721727    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721730    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721733    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721736    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721739    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721742    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721744    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721747    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721750    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721752    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721755    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.721758    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722479    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722486    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722489    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722493    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722496    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.723652 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722499    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722502    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722505    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722507    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722511    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722513    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722516    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722519    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722522    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722524    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722527    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722530    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722533    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722536    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722539    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722541    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722544    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722546    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722549    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722552    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.724180 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722556    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722560    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722563    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722566    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722569    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722571    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722574    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722576    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722579    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722582    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722584    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722588    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722591    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722594    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722597    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722600    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722603    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722605    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722608    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722613    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.724702 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722616    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722619    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722622    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722625    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722627    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722631    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722633    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722636    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722638    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722641    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722645    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722649    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722652    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722654    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722657    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722660    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722662    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722665    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722668    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.725205 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722670    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722673    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722676    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722679    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722682    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722685    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722687    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722690    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722694    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722696    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722699    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722702    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722705    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722707    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722710    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722713    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722715    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722718    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722721    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722724    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.725695 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722727    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722729    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722732    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722734    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722737    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722739    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722742    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722745    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722747    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722750    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722753    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722756    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722758    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722761    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.722764    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724013    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724021    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724024    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724028    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724030    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.726174 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724033    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724036    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724038    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724041    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724044    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724047    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724049    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724052    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724054    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724058    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724062    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724066    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724069    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724072    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724075    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724078    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724080    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724083    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724086    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.726765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724089    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724092    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724096    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724098    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724101    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724104    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724107    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724109    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724112    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724114    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724117    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724119    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724122    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724124    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724127    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724130    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724133    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724135    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724138    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.727237 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724140    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724144    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724148    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724151    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724153    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724156    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724159    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724162    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724165    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724167    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724173    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724176    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724178    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724181    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724185    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724188    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724191    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724193    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724196    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.727727 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724198    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724201    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724204    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724206    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724208    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724211    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724213    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724216    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724218    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724221    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724223    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724226    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724228    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724231    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724233    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724235    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724238    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724240    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724243    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724245    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.728190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724248    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724251    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724254    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724271    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724275    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724277    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724280    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724283    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724286    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724288    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724291    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724294    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724296    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724299    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724302    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724304    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.724307    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724377    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724408    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724419    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724423    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 16:17:06.728685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724428    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724431    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724435    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724439    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724442    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724445    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724449    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724452    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724456    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724458    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724461    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724465    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724468    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724472    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724476    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724479    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724483    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724486    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724489    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724493    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724496    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724499    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724502    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724506    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 16:17:06.729222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724508    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724511    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724514    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724517    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724521    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724524    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724528    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724530    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724534    2577 flags.go:64] FLAG: --enable-server="true"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724536    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724541    2577 flags.go:64] FLAG: --event-burst="100"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724545    2577 flags.go:64] FLAG: --event-qps="50"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724547    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724550    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724553    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724557    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724560    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724563    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724566    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724569    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724572    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724577    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724580    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724583    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724586    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 16:17:06.729818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724589    2577 flags.go:64] FLAG: --feature-gates=""
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724593    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724596    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724599    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724602    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724605    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724608    2577 flags.go:64] FLAG: --help="false"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724611    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724614    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724617    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724619    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724623    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724626    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724629    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724632    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724635    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724638    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724641    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724645    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724648    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724651    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724654    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724657    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724659    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 16:17:06.730459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724662    2577 flags.go:64] FLAG: --lock-file=""
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724665    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724667    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724670    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724677    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724681    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724684    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724686    2577 flags.go:64] FLAG: --logging-format="text"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724689    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724693    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724696    2577 flags.go:64] FLAG: --manifest-url=""
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724698    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724703    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724707    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724711    2577 flags.go:64] FLAG: --max-pods="110"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724714    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724717    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724721    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724723    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724726    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724729    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724732    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724740    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724743    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724746    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 16:17:06.731040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724749    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724753    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724759    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724762    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724765    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724768    2577 flags.go:64] FLAG: --port="10250"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724771    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724774    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-068e75fb9d19a4841"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724777    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724780    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724783    2577 flags.go:64] FLAG: --register-node="true"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724786    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724790    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724794    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724796    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724799    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724803    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724806    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724809    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724812    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724814    2577 flags.go:64] FLAG: --runonce="false"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724817    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724820    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724823    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724826    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 16:17:06.731679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724829    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724831    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724835    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724838    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724841    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724844    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724846    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724849    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724853    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724856    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724859    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724864    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724867    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724870    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724874    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724876    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724879    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724882    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724885    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724888    2577 flags.go:64] FLAG: --v="2"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724891    2577 flags.go:64] FLAG: --version="false"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724896    2577 flags.go:64] FLAG: --vmodule=""
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724899    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.724902    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725015    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.732399 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725019    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725022    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725025    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725028    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725032    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725034    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725037    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725040    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725042    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725044    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725047    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725050    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725052    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725054    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725057    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725061    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725063    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725066    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725068    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725071    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.733536 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725074    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725076    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725079    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725081    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725084    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725088    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725091    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725094    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725097    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725100    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725103    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725106    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725109    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725111    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725114    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725117    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725121    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725124    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725127    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.734404 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725129    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725132    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725134    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725137    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725139    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725142    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725144    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725147    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725152    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725154    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725157    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725160    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725162    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725164    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725167    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725170    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725172    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725175    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725177    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.734931 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725180    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725183    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725186    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725190    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725193    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725196    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725199    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725202    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725205    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725208    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725211    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725214    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725217    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725219    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725222    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725224    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725227    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725229    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725232    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.735564 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725234    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725237    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725241    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725244    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725246    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725249    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725251    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725254    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725271    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725275    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725279    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725283    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725287    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725292    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725294    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725297    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725300    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725302    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725306    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725309    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.736432 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.725319    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.726015    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.734463    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.734482    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.735339    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.735354    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735440    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735449    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735454    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735460    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735464    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735469    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735473    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735481    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.737083 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735489    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735495    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735500    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735504    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735508    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735513    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735517    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735521    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735526    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735531    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735535    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735539    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735543    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735547    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735552    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735556    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735560    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735564    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735568    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735573    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.737495 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735577    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735584    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735589    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735594    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735598    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735602    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735607    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735611    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735615    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735620    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735624    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735628    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735632    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735636    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735640    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735645    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735649    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735656    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735660    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735664    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.738371 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735668    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735672    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735676    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735681    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735686    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735690    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735694    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735699    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735703    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735707    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735712    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735716    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735720    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735725    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735730    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735735    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735739    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735743    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735748    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735752    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.739090 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735756    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735760    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735764    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735768    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735772    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735777    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735781    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735785    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735796    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735801    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735805    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735811    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735817    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735822    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735826    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735831    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735836    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735841    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735846    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.739630 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735851    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735855    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735859    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735863    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735868    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735872    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735877    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735881    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735886    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735890    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735895    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.735899    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.735907    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736085    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 16:17:06.740198 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736094    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736098    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736102    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736106    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736111    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736115    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736119    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736123    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736127    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736131    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736138    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736144    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736148    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736152    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736157    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736162    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736166    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736170    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736174    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736178    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 16:17:06.740646 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736182    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736187    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736191    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736196    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736200    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736204    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736209    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736214    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736218    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736222    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736227    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736231    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736235    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736239    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736243    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736247    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736252    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736276    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736281    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736286    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 16:17:06.741192 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736290    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736294    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736298    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736302    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736306    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736313    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736319    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736324    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736329    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736334    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736338    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736343    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736347    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736352    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736357    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736361    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736365    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736370    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736374    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 16:17:06.741718 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736379    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736384    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736388    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736393    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736398    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736402    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736407    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736411    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736415    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736419    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736424    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736428    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736432    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736436    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736440    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736444    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736448    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736452    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736456    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736460    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 16:17:06.742229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736464    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736469    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736474    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736478    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736482    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736486    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736491    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736495    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736499    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736504    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736508    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736512    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736516    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736520    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736526    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736530    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736534    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736539    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 16:17:06.742771 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:06.736542    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.736551    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.736712    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.739673    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.740847    2577 server.go:1019] "Starting client certificate rotation"
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.740956    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 16:17:06.743208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.741009    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 16:17:06.744618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.744601    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 16:17:06.747594 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.746102    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 16:17:06.759733 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.759716    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 10 16:17:06.763141 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.763127    2577 log.go:25] "Validated CRI v1 image API"
Sep 10 16:17:06.764067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.764050    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 16:17:06.766667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.766647    2577 fs.go:135] Filesystem UUIDs: map[66cfa845-fccf-4eb8-9714-89b489891cae:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 fb420229-f1f6-4829-96d3-f97ea058dda0:/dev/nvme0n1p4]
Sep 10 16:17:06.766720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.766667    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 16:17:06.771904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.771792    2577 manager.go:216] Machine: {Timestamp:2026-09-10 16:17:06.770275282 +0000 UTC m=+0.368092734 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100624 MemoryCapacity:33164484608 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ea3679a4bec2df26d5038701aa5c5 SystemUUID:ec2ea367-9a4b-ec2d-f26d-5038701aa5c5 BootID:662a75d2-9d13-489d-86a9-f976ce8c1c33 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0f:80:df:dc:f9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0f:80:df:dc:f9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:8a:ca:e3:8f:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164484608 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 16:17:06.771904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.771898    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 16:17:06.772001 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.771976    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 16:17:06.775349 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.775324    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 16:17:06.775498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.775353    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-40.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 16:17:06.776156 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.776144    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 16:17:06.776201 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.776159    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 16:17:06.776201 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.776180    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 16:17:06.776271 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.776225    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 16:17:06.776986 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.776975    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 16:17:06.778873 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.778861    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 16:17:06.783630 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783617    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 16:17:06.783749 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783739    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 16:17:06.783779 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783771    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 16:17:06.783813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783783    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 16:17:06.783813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783806    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 16:17:06.783899 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783817    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 10 16:17:06.783899 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.783835    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 16:17:06.784296 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.784282    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 16:17:06.784326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.784307    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 16:17:06.786693 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.786672    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 16:17:06.789798 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789783    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 16:17:06.789900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789813    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 16:17:06.789977 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789967    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 16:17:06.790008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789982    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 16:17:06.790008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789988    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 16:17:06.790008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.789993    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 16:17:06.790008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790001    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 16:17:06.790008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790006    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 16:17:06.790718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790700    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 16:17:06.790718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790711    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 16:17:06.790834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790721    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 16:17:06.790834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790730    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 16:17:06.790834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790742    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 16:17:06.790834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.790751    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 16:17:06.791649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.791634    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 16:17:06.791649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.791647    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 16:17:06.793008 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.792984    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 16:17:06.793008 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.792983    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 16:17:06.793099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.793026    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-40.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 16:17:06.795642 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.795628    2577 server.go:1268] "Started kubelet"
Sep 10 16:17:06.795754 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.795695    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 16:17:06.795805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.795773    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 16:17:06.796111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.796078    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 16:17:06.796517 ip-10-0-136-40 systemd[1]: Started Kubernetes Kubelet.
Sep 10 16:17:06.800588 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.800510    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 16:17:06.803164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.803147    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 16:17:06.803759 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.803741    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 16:17:06.808151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.808130    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 16:17:06.808968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.808946    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 16:17:06.809128 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.809105    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:06.809759 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809740    2577 factory.go:55] Registering systemd factory
Sep 10 16:17:06.809759 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809751    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 16:17:06.809885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809760    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 10 16:17:06.809885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809749    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 16:17:06.809885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809824    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 16:17:06.809992 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809921    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 16:17:06.809992 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.809929    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 16:17:06.810050 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810022    2577 factory.go:153] Registering CRI-O factory
Sep 10 16:17:06.810050 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810034    2577 factory.go:223] Registration of the crio container factory successfully
Sep 10 16:17:06.810105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810078    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 16:17:06.810105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810092    2577 factory.go:103] Registering Raw factory
Sep 10 16:17:06.810105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810102    2577 manager.go:1196] Started watching for new ooms in manager
Sep 10 16:17:06.810450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.810439    2577 manager.go:349] Starting recovery of all containers
Sep 10 16:17:06.815784 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.815754    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 16:17:06.815893 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.815870    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-40.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 16:17:06.817049 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.815779    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18d401e31511741a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-09-10 16:17:06.795598874 +0000 UTC m=+0.393416326,LastTimestamp:2026-09-10 16:17:06.795598874 +0000 UTC m=+0.393416326,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Sep 10 16:17:06.820504 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.820103    2577 manager.go:354] Recovery completed
Sep 10 16:17:06.826033 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.826018    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:06.827665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.827636    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qfx7k"
Sep 10 16:17:06.827904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.827884    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:06.827978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.827925    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:06.827978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.827941    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:06.828604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.828582    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 16:17:06.828604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.828600    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 16:17:06.828737 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.828605    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:06.828737 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.828620    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 16:17:06.829334 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.829318    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:06.829424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.829356    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:06.829424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.829378    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:06.829682 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.829589    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-40.ec2.internal.18d401e316fe64b7  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-40.ec2.internal,UID:ip-10-0-136-40.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-136-40.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-136-40.ec2.internal,},FirstTimestamp:2026-09-10 16:17:06.827904183 +0000 UTC m=+0.425721642,LastTimestamp:2026-09-10 16:17:06.827904183 +0000 UTC m=+0.425721642,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-40.ec2.internal,}"
Sep 10 16:17:06.830921 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.830909    2577 policy_none.go:49] "None policy: Start"
Sep 10 16:17:06.830963 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.830925    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 16:17:06.830963 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.830935    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 16:17:06.832877 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.832866    2577 policy_none.go:47] "Start"
Sep 10 16:17:06.835586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.835570    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qfx7k"
Sep 10 16:17:06.908512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.908470    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 16:17:06.910571 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.909659    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 16:17:06.910571 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.909691    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:06.910571 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.909701    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 16:17:06.910571 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.909737    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 16:17:06.910571 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:06.909832    2577 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 10 16:17:06.912285 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:06.912248    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:07.007958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.007931    2577 manager.go:337] "Starting Device Plugin manager"
Sep 10 16:17:07.008101 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.007986    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 16:17:07.008101 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.007999    2577 server.go:87] "Starting device plugin registration server"
Sep 10 16:17:07.008411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.008397    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 16:17:07.008496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.008412    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 16:17:07.008566 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.008505    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 16:17:07.008650 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.008638    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 16:17:07.008703 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.008652    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 16:17:07.009186 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.009166    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 16:17:07.009291 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.009218    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:07.010294 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.010276    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"]
Sep 10 16:17:07.010350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.010340    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:07.011766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.011750    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:07.011862 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.011779    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:07.011862 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.011795    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:07.014119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014107    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:07.014313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014293    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.014361 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014328    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:07.014940 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014924    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:07.015042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014923    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:07.015042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014953    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:07.015042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014967    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:07.015042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014971    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:07.015042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.014984    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:07.017647 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.017632    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.017725 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.017660    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:07.018366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.018352    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:07.018433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.018380    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:07.018433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.018391    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:07.023408 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.023391    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-40.ec2.internal\" not found" node="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.044016 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.043992    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-40.ec2.internal\" not found" node="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.048604 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.048589    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-40.ec2.internal\" not found" node="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.108787 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.108753    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 16:17:07.109785 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.109761    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 16:17:07.109904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.109794    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 16:17:07.109904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.109804    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeHasSufficientPID"
Sep 10 16:17:07.109904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.109833    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.111014 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.110995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.111067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.111022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.111067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.111040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/251c9980d3ffefc3102b5100c4ca2cb5-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"251c9980d3ffefc3102b5100c4ca2cb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.119033 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.119010    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.119132 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.119040    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-40.ec2.internal\": node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:07.148988 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.148953    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:07.211171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.211171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211171    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.211357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/251c9980d3ffefc3102b5100c4ca2cb5-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"251c9980d3ffefc3102b5100c4ca2cb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.211357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.211357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211292    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e548e3497cb82467734f20174cf45c0-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal\" (UID: \"4e548e3497cb82467734f20174cf45c0\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.211357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.211297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/251c9980d3ffefc3102b5100c4ca2cb5-config\") pod \"kube-apiserver-proxy-ip-10-0-136-40.ec2.internal\" (UID: \"251c9980d3ffefc3102b5100c4ca2cb5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.249504 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.249440    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:07.345994 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.345952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:07.349530 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:07.349501    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:07.351664 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:07.351648    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:08.646350 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:08.646314    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:08.746915 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:08.746855    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:08.833850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:08.833813    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:08.847874 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:08.847851    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:08.936397 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:08.936372    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 16:17:08.936766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:08.936508    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 16:17:08.936766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:08.936503    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 16:17:08.948670 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:08.948649    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:09.035141 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.035036    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 16:12:08 +0000 UTC" deadline="2028-03-23 09:10:24.367521005 +0000 UTC"
Sep 10 16:17:09.035141 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.035086    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13432h53m15.332437882s"
Sep 10 16:17:09.049219 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:09.049185    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:09.150181 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:09.150160    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:09.250740 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:09.250708    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-40.ec2.internal\" not found"
Sep 10 16:17:09.278858 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:09.278818    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod251c9980d3ffefc3102b5100c4ca2cb5.slice/crio-04d95a457900d7d0f4e296ed3bb442e1d808d32d8ead4ea46bf081533c829450 WatchSource:0}: Error finding container 04d95a457900d7d0f4e296ed3bb442e1d808d32d8ead4ea46bf081533c829450: Status 404 returned error can't find the container with id 04d95a457900d7d0f4e296ed3bb442e1d808d32d8ead4ea46bf081533c829450
Sep 10 16:17:09.279694 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:09.279657    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e548e3497cb82467734f20174cf45c0.slice/crio-eafbd1e222104b34fb64167c4570c3bfa7905022e9d46b4725d4d07f1ccae6c5 WatchSource:0}: Error finding container eafbd1e222104b34fb64167c4570c3bfa7905022e9d46b4725d4d07f1ccae6c5: Status 404 returned error can't find the container with id eafbd1e222104b34fb64167c4570c3bfa7905022e9d46b4725d4d07f1ccae6c5
Sep 10 16:17:09.285111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.285094    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:17:09.306449 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.306418    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:09.338864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.338838    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:09.405218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.405187    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:09.420001 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.419980    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 16:17:09.421052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.421037    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal"
Sep 10 16:17:09.428229 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.428212    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 16:17:09.981362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.981327    2577 apiserver.go:52] "Watching apiserver"
Sep 10 16:17:09.994375 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.994348    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 16:17:09.994790 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.994764    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal","openshift-cluster-node-tuning-operator/tuned-9jvcb","openshift-network-diagnostics/network-check-target-sdlsb","openshift-ovn-kubernetes/ovnkube-node-n746n","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4","openshift-dns/node-resolver-r84rk","openshift-image-registry/node-ca-d24lj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal","openshift-multus/multus-95whm","openshift-multus/multus-additional-cni-plugins-hr6l7","openshift-multus/network-metrics-daemon-87mpg","openshift-network-operator/iptables-alerter-k4r9t","kube-system/global-pull-secret-syncer-xwrj7","kube-system/konnectivity-agent-w27lk"]
Sep 10 16:17:09.997483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.997460    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:09.999580 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:09.999563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:09.999669 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:09.999630    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:10.000258 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.000238    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.000503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.000487    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r7t2d\""
Sep 10 16:17:10.000711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.000679    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 16:17:10.001021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.001001    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.005326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.004683    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 16:17:10.005326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.005263    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:10.005471 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.005343    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:10.008775 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.008752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.011929 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.011909    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 16:17:10.012020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.011947    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 16:17:10.012265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.011913    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-c5t4r\""
Sep 10 16:17:10.012483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.012460    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 16:17:10.012599 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.012581    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.012660 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.012651    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.012767 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.012752    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 16:17:10.012998 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.012980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovn-node-metrics-cert\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-systemd-units\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-log-socket\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-netd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-config\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.014980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfzfs\" (UniqueName: \"kubernetes.io/projected/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-kube-api-access-pfzfs\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015010    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-var-lib-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015033    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-etc-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1500121e-5856-4e8a-96a1-a7815dc5905c-serviceca\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015150    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-ovn\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-node-log\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.017423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015214    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-slash\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzz54\" (UniqueName: \"kubernetes.io/projected/1500121e-5856-4e8a-96a1-a7815dc5905c-kube-api-access-kzz54\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015317    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-netns\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015339    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-bin\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1500121e-5856-4e8a-96a1-a7815dc5905c-host\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015384    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-dbus\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-kubelet\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-systemd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-script-lib\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015492    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-kubelet-config\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.015513    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-env-overrides\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.016051    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.016354    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8llsm\""
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.016532    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.016743    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.016973    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.017340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.018240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.017848    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.019706 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.019684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.020778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.020758    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pbklj\""
Sep 10 16:17:10.021062 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021042    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.021151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021076    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.021316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021301    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 16:17:10.021387 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021345    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.021443 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021306    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p25v8\""
Sep 10 16:17:10.021640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021506    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.022649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.021858    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 16:17:10.022649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.022141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.022649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.022598    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 16:17:10.023789 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.023251    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4vz7z\""
Sep 10 16:17:10.023789 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.023333    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 16:17:10.024456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.024055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.025171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.024748    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.025171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.024763    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-gqgs9\""
Sep 10 16:17:10.025171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.025026    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.027085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.027066    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 16:17:10.027172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.027095    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 16:17:10.027172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.027150    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sqrnb\""
Sep 10 16:17:10.030048 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.030028    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.030133 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.030108    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:10.033926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.033907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.037882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.037546    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:10.037882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.037738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hmg75\""
Sep 10 16:17:10.038026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.037906    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:10.038076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.038064    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 16:17:10.039182 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.039162    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ql9ks"
Sep 10 16:17:10.048767 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.048746    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ql9ks"
Sep 10 16:17:10.107412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.107377    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 16:17:10.112198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.112137    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" event={"ID":"251c9980d3ffefc3102b5100c4ca2cb5","Type":"ContainerStarted","Data":"04d95a457900d7d0f4e296ed3bb442e1d808d32d8ead4ea46bf081533c829450"}
Sep 10 16:17:10.113849 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.113820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"4e548e3497cb82467734f20174cf45c0","Type":"ContainerStarted","Data":"eafbd1e222104b34fb64167c4570c3bfa7905022e9d46b4725d4d07f1ccae6c5"}
Sep 10 16:17:10.116699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116672    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovn-node-metrics-cert\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.116805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-log-socket\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.116805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116740    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-netd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.116805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-config\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.116805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfzfs\" (UniqueName: \"kubernetes.io/projected/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-kube-api-access-pfzfs\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-sys-fs\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116839    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-host\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116861    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab87d95-1588-4a35-9211-d0d53b937dd2-tmp-dir\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116886    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-var-lib-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-cnibin\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-system-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.116984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-socket-dir-parent\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-bin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.117045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-etc-kubernetes\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1500121e-5856-4e8a-96a1-a7815dc5905c-serviceca\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-ovn\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-node-log\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-tmp\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-var-lib-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72045530-1543-405a-9763-852fb5e9c22f-konnectivity-ca\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117265    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-log-socket\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-slash\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzz54\" (UniqueName: \"kubernetes.io/projected/1500121e-5856-4e8a-96a1-a7815dc5905c-kube-api-access-kzz54\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117376    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-netns\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117430    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117455    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-modprobe-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-netd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkcv9\" (UniqueName: \"kubernetes.io/projected/9ab87d95-1588-4a35-9211-d0d53b937dd2-kube-api-access-bkcv9\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.117523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-slash\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-dbus\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-dbus\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-run-netns\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-ovn\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-kubelet\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117735    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-node-log\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117755    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-kubelet\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-etc-selinux\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117823    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1500121e-5856-4e8a-96a1-a7815dc5905c-serviceca\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117843    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj2zq\" (UniqueName: \"kubernetes.io/projected/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-kube-api-access-nj2zq\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117912    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ab87d95-1588-4a35-9211-d0d53b937dd2-hosts-file\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117934    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-multus\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-systemd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.117999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-script-lib\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.118407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-registration-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-systemd\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysconfig\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-run\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118137    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-sys\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-config\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-tuned\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqhx2\" (UniqueName: \"kubernetes.io/projected/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-kube-api-access-fqhx2\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-kubelet-config\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-env-overrides\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-device-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118358    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-kubelet-config\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-netns\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118448    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-systemd-units\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.118481    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-systemd-units\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.119192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.118560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:10.618521072 +0000 UTC m=+3.020322180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovnkube-script-lib\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-466tz\" (UniqueName: \"kubernetes.io/projected/534d15f4-9147-439b-8a20-9d951e5cf6f3-kube-api-access-466tz\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118664    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-kubernetes\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118688    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-host-slash\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118711    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-env-overrides\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118712    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-conf-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-etc-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-etc-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-systemd\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118856    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cni-binary-copy\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-run-openvswitch\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-kubelet\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-multus-certs\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118946    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-socket-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.118982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-var-lib-kubelet\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-k8s-cni-cncf-io\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-daemon-config\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119071    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72045530-1543-405a-9763-852fb5e9c22f-agent-certs\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-bin\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-iptables-alerter-script\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xk47t\" (UniqueName: \"kubernetes.io/projected/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-kube-api-access-xk47t\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-host-cni-bin\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ct4m\" (UniqueName: \"kubernetes.io/projected/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-kube-api-access-5ct4m\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119260    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1500121e-5856-4e8a-96a1-a7815dc5905c-host\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1500121e-5856-4e8a-96a1-a7815dc5905c-host\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stjvn\" (UniqueName: \"kubernetes.io/projected/6b7858a1-8d78-4152-8162-0555789ff294-kube-api-access-stjvn\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.120843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-lib-modules\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119499    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-conf\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119552    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cnibin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.121651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.119571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-hostroot\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.122777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.122754    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-ovn-node-metrics-cert\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.125634 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.125614    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:10.125732 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.125640    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:10.125732 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.125660    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:10.125732 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.125721    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:10.62569756 +0000 UTC m=+3.027498680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:10.128415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.128390    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfzfs\" (UniqueName: \"kubernetes.io/projected/89b9e2a7-13a7-40eb-9194-06c2f9bad9a2-kube-api-access-pfzfs\") pod \"ovnkube-node-n746n\" (UID: \"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2\") " pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.128575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.128554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzz54\" (UniqueName: \"kubernetes.io/projected/1500121e-5856-4e8a-96a1-a7815dc5905c-kube-api-access-kzz54\") pod \"node-ca-d24lj\" (UID: \"1500121e-5856-4e8a-96a1-a7815dc5905c\") " pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.215431 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.215389    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:10.220836 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.220970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-modprobe-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.220970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220873    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkcv9\" (UniqueName: \"kubernetes.io/projected/9ab87d95-1588-4a35-9211-d0d53b937dd2-kube-api-access-bkcv9\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.220970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.220970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-etc-selinux\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.220975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj2zq\" (UniqueName: \"kubernetes.io/projected/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-kube-api-access-nj2zq\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221000    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ab87d95-1588-4a35-9211-d0d53b937dd2-hosts-file\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-multus\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-registration-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysconfig\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221109    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-etc-selinux\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221141    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-modprobe-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-run\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-run\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.221214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221213    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-kubelet-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-sys\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221255    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-sys\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221256    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-tuned\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqhx2\" (UniqueName: \"kubernetes.io/projected/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-kube-api-access-fqhx2\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-device-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221340    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-multus\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-netns\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-466tz\" (UniqueName: \"kubernetes.io/projected/534d15f4-9147-439b-8a20-9d951e5cf6f3-kube-api-access-466tz\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-device-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.221421    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-kubernetes\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-registration-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221453    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-host-slash\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysconfig\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ab87d95-1588-4a35-9211-d0d53b937dd2-hosts-file\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221519    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-conf-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.221780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-host-slash\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.221535    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:10.721514093 +0000 UTC m=+3.123315212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-netns\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-systemd\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cni-binary-copy\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221617    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-kubernetes\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-kubelet\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-conf-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-multus-certs\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221690    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-socket-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-systemd\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-multus-certs\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-kubelet\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-var-lib-kubelet\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221760    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-k8s-cni-cncf-io\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221773    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-var-lib-kubelet\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.222633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221785    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-daemon-config\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72045530-1543-405a-9763-852fb5e9c22f-agent-certs\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-run-k8s-cni-cncf-io\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221833    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-socket-dir\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-iptables-alerter-script\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xk47t\" (UniqueName: \"kubernetes.io/projected/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-kube-api-access-xk47t\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221917    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ct4m\" (UniqueName: \"kubernetes.io/projected/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-kube-api-access-5ct4m\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stjvn\" (UniqueName: \"kubernetes.io/projected/6b7858a1-8d78-4152-8162-0555789ff294-kube-api-access-stjvn\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221957    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-d\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.221966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-lib-modules\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222084    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-conf\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-lib-modules\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222110    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cnibin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.223521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-hostroot\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-sys-fs\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-host\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab87d95-1588-4a35-9211-d0d53b937dd2-tmp-dir\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-cnibin\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222263    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-system-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222308    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-socket-dir-parent\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cni-binary-copy\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-bin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-daemon-config\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-etc-kubernetes\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-tmp\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72045530-1543-405a-9763-852fb5e9c22f-konnectivity-ca\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222466    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-host-var-lib-cni-bin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b7858a1-8d78-4152-8162-0555789ff294-sys-fs\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222728    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-etc-kubernetes\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222726    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-host\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.224348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-cnibin\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-system-cni-dir\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.222887    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-multus-socket-dir-parent\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab87d95-1588-4a35-9211-d0d53b937dd2-tmp-dir\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/534d15f4-9147-439b-8a20-9d951e5cf6f3-system-cni-dir\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-sysctl-conf\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-cnibin\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-hostroot\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223367    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-cni-binary-copy\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/72045530-1543-405a-9763-852fb5e9c22f-konnectivity-ca\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-iptables-alerter-script\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.223865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/534d15f4-9147-439b-8a20-9d951e5cf6f3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.225179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.224481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-etc-tuned\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.226359 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.226336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/72045530-1543-405a-9763-852fb5e9c22f-agent-certs\") pod \"konnectivity-agent-w27lk\" (UID: \"72045530-1543-405a-9763-852fb5e9c22f\") " pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.226561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.226541    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-tmp\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.235118 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.234989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqhx2\" (UniqueName: \"kubernetes.io/projected/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-kube-api-access-fqhx2\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.236218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.236195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj2zq\" (UniqueName: \"kubernetes.io/projected/ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53-kube-api-access-nj2zq\") pod \"tuned-9jvcb\" (UID: \"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53\") " pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.236327 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.236201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkcv9\" (UniqueName: \"kubernetes.io/projected/9ab87d95-1588-4a35-9211-d0d53b937dd2-kube-api-access-bkcv9\") pod \"node-resolver-r84rk\" (UID: \"9ab87d95-1588-4a35-9211-d0d53b937dd2\") " pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.236399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.236321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ct4m\" (UniqueName: \"kubernetes.io/projected/dc1d39a1-3c08-48dd-84b3-ff9ebbf63831-kube-api-access-5ct4m\") pod \"multus-95whm\" (UID: \"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831\") " pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.236546 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.236521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stjvn\" (UniqueName: \"kubernetes.io/projected/6b7858a1-8d78-4152-8162-0555789ff294-kube-api-access-stjvn\") pod \"aws-ebs-csi-driver-node-djbs4\" (UID: \"6b7858a1-8d78-4152-8162-0555789ff294\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.237619 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.237564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xk47t\" (UniqueName: \"kubernetes.io/projected/9780784f-b25a-4cb6-9ee6-54cb59bf3a60-kube-api-access-xk47t\") pod \"iptables-alerter-k4r9t\" (UID: \"9780784f-b25a-4cb6-9ee6-54cb59bf3a60\") " pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.237861 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.237839    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-466tz\" (UniqueName: \"kubernetes.io/projected/534d15f4-9147-439b-8a20-9d951e5cf6f3-kube-api-access-466tz\") pod \"multus-additional-cni-plugins-hr6l7\" (UID: \"534d15f4-9147-439b-8a20-9d951e5cf6f3\") " pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.311022 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.310986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d24lj"
Sep 10 16:17:10.323701 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.323671    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1500121e_5856_4e8a_96a1_a7815dc5905c.slice/crio-fb99d0b9ca95512b28110b4385e747adf2a8ca2266328ba70390cb1be6e8569d WatchSource:0}: Error finding container fb99d0b9ca95512b28110b4385e747adf2a8ca2266328ba70390cb1be6e8569d: Status 404 returned error can't find the container with id fb99d0b9ca95512b28110b4385e747adf2a8ca2266328ba70390cb1be6e8569d
Sep 10 16:17:10.324608 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.324582    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:10.332298 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.332266    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89b9e2a7_13a7_40eb_9194_06c2f9bad9a2.slice/crio-7140abeb7aec881fe68d08b8af1328bcd062d732333d1b3af86e3b9e1ca6dccc WatchSource:0}: Error finding container 7140abeb7aec881fe68d08b8af1328bcd062d732333d1b3af86e3b9e1ca6dccc: Status 404 returned error can't find the container with id 7140abeb7aec881fe68d08b8af1328bcd062d732333d1b3af86e3b9e1ca6dccc
Sep 10 16:17:10.335253 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.335241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4"
Sep 10 16:17:10.341973 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.341952    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hr6l7"
Sep 10 16:17:10.343853 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.343830    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b7858a1_8d78_4152_8162_0555789ff294.slice/crio-c40684757df656b76c16da788ba774716c6e6b8d56a76d7a72e95b65081a56a1 WatchSource:0}: Error finding container c40684757df656b76c16da788ba774716c6e6b8d56a76d7a72e95b65081a56a1: Status 404 returned error can't find the container with id c40684757df656b76c16da788ba774716c6e6b8d56a76d7a72e95b65081a56a1
Sep 10 16:17:10.350205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.350169    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-95whm"
Sep 10 16:17:10.353892 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.353868    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534d15f4_9147_439b_8a20_9d951e5cf6f3.slice/crio-6aedf082f218a1b86455263cf4101cdcd599287b6f658c1dd3254cb407b8bcf6 WatchSource:0}: Error finding container 6aedf082f218a1b86455263cf4101cdcd599287b6f658c1dd3254cb407b8bcf6: Status 404 returned error can't find the container with id 6aedf082f218a1b86455263cf4101cdcd599287b6f658c1dd3254cb407b8bcf6
Sep 10 16:17:10.357202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.357176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-r84rk"
Sep 10 16:17:10.359211 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.359191    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc1d39a1_3c08_48dd_84b3_ff9ebbf63831.slice/crio-b4fe4d1df0560d1ccdbd3ef0509bf8c49758d1fc763dc0e8ce2b343c5bfd36c0 WatchSource:0}: Error finding container b4fe4d1df0560d1ccdbd3ef0509bf8c49758d1fc763dc0e8ce2b343c5bfd36c0: Status 404 returned error can't find the container with id b4fe4d1df0560d1ccdbd3ef0509bf8c49758d1fc763dc0e8ce2b343c5bfd36c0
Sep 10 16:17:10.367235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.367086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:10.367793 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.367770    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab87d95_1588_4a35_9211_d0d53b937dd2.slice/crio-355cb1e378ffe068fa8100fedd89743bea32f13c200a2b41d3981bf6fb94e52b WatchSource:0}: Error finding container 355cb1e378ffe068fa8100fedd89743bea32f13c200a2b41d3981bf6fb94e52b: Status 404 returned error can't find the container with id 355cb1e378ffe068fa8100fedd89743bea32f13c200a2b41d3981bf6fb94e52b
Sep 10 16:17:10.376452 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.376421    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72045530_1543_405a_9763_852fb5e9c22f.slice/crio-535541d8262d146b67efeeef39d41ee39291e9e8da41ef20d7751f11ec67681b WatchSource:0}: Error finding container 535541d8262d146b67efeeef39d41ee39291e9e8da41ef20d7751f11ec67681b: Status 404 returned error can't find the container with id 535541d8262d146b67efeeef39d41ee39291e9e8da41ef20d7751f11ec67681b
Sep 10 16:17:10.376613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.376544    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9jvcb"
Sep 10 16:17:10.383221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.383198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4r9t"
Sep 10 16:17:10.388295 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.388072    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6a08bb_26c2_4a3a_a9c4_1ca373eafc53.slice/crio-04069967e52eff7e81bb4eb3eeb97dabace90316ad41ed14fe9c3a9cf3319878 WatchSource:0}: Error finding container 04069967e52eff7e81bb4eb3eeb97dabace90316ad41ed14fe9c3a9cf3319878: Status 404 returned error can't find the container with id 04069967e52eff7e81bb4eb3eeb97dabace90316ad41ed14fe9c3a9cf3319878
Sep 10 16:17:10.393424 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:10.393393    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9780784f_b25a_4cb6_9ee6_54cb59bf3a60.slice/crio-8cca5fdee26ddef6607ddbadb5f4c40e24fbb523f6149bfd4cde695b651ce209 WatchSource:0}: Error finding container 8cca5fdee26ddef6607ddbadb5f4c40e24fbb523f6149bfd4cde695b651ce209: Status 404 returned error can't find the container with id 8cca5fdee26ddef6607ddbadb5f4c40e24fbb523f6149bfd4cde695b651ce209
Sep 10 16:17:10.626017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.625943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:10.626017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.625985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:10.626201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626100    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:10.626201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626140    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:10.626201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626158    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:10.626201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626172    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:10.626201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626173    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:11.626152405 +0000 UTC m=+4.027953530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:10.626465 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.626221    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:11.626206286 +0000 UTC m=+4.028007395 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:10.726535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:10.726505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:10.726714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.726622    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:10.726714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:10.726690    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:11.72667498 +0000 UTC m=+4.128476107 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:11.052602 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.052502    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 16:12:10 +0000 UTC" deadline="2028-02-18 16:29:40.842272325 +0000 UTC"
Sep 10 16:17:11.052602 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.052556    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12624h12m29.789721454s"
Sep 10 16:17:11.106880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.106853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:11.107046 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.106975    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:11.116725 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.116700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d24lj" event={"ID":"1500121e-5856-4e8a-96a1-a7815dc5905c","Type":"ContainerStarted","Data":"fb99d0b9ca95512b28110b4385e747adf2a8ca2266328ba70390cb1be6e8569d"}
Sep 10 16:17:11.118727 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.118483    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jvcb" event={"ID":"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53","Type":"ContainerStarted","Data":"04069967e52eff7e81bb4eb3eeb97dabace90316ad41ed14fe9c3a9cf3319878"}
Sep 10 16:17:11.122814 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.122791    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w27lk" event={"ID":"72045530-1543-405a-9763-852fb5e9c22f","Type":"ContainerStarted","Data":"535541d8262d146b67efeeef39d41ee39291e9e8da41ef20d7751f11ec67681b"}
Sep 10 16:17:11.124164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.124138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4" event={"ID":"6b7858a1-8d78-4152-8162-0555789ff294","Type":"ContainerStarted","Data":"c40684757df656b76c16da788ba774716c6e6b8d56a76d7a72e95b65081a56a1"}
Sep 10 16:17:11.125433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.125401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"7140abeb7aec881fe68d08b8af1328bcd062d732333d1b3af86e3b9e1ca6dccc"}
Sep 10 16:17:11.126649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.126624    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4r9t" event={"ID":"9780784f-b25a-4cb6-9ee6-54cb59bf3a60","Type":"ContainerStarted","Data":"8cca5fdee26ddef6607ddbadb5f4c40e24fbb523f6149bfd4cde695b651ce209"}
Sep 10 16:17:11.127933 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.127908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r84rk" event={"ID":"9ab87d95-1588-4a35-9211-d0d53b937dd2","Type":"ContainerStarted","Data":"355cb1e378ffe068fa8100fedd89743bea32f13c200a2b41d3981bf6fb94e52b"}
Sep 10 16:17:11.129149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.129125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95whm" event={"ID":"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831","Type":"ContainerStarted","Data":"b4fe4d1df0560d1ccdbd3ef0509bf8c49758d1fc763dc0e8ce2b343c5bfd36c0"}
Sep 10 16:17:11.130336 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.130315    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerStarted","Data":"6aedf082f218a1b86455263cf4101cdcd599287b6f658c1dd3254cb407b8bcf6"}
Sep 10 16:17:11.323396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.323318    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 16:17:11.634467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.634391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:11.634467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.634460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:11.634719 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.634639    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:11.634719 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.634663    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:11.634719 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.634676    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:11.634880 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.634733    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:13.634715177 +0000 UTC m=+6.036516284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:11.635135 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.635106    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:11.635201 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.635172    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:13.635157211 +0000 UTC m=+6.036958320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:11.735065 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:11.735028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:11.735232 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.735208    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:11.735310 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:11.735270    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:13.735252775 +0000 UTC m=+6.137053883 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:12.052985 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.052914    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 16:12:10 +0000 UTC" deadline="2028-02-04 02:23:00.389684361 +0000 UTC"
Sep 10 16:17:12.052985 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.052970    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12274h5m48.336718545s"
Sep 10 16:17:12.108243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.107833    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:12.108243 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:12.107976    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:12.109771 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.109745    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:12.109888 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:12.109867    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:12.145782 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.145702    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" event={"ID":"251c9980d3ffefc3102b5100c4ca2cb5","Type":"ContainerStarted","Data":"5e9ad86b870180a4f66e5efe42495b9ed494afafd4cfbb5fe2fa20a463801e94"}
Sep 10 16:17:12.158298 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.158244    2577 generic.go:356] "Generic (PLEG): container finished" podID="4e548e3497cb82467734f20174cf45c0" containerID="3770bf76b89b9ecb603a450a140b498c5ab210a702c8dafc26fe9106dd0c3269" exitCode=0
Sep 10 16:17:12.158473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.158330    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"4e548e3497cb82467734f20174cf45c0","Type":"ContainerDied","Data":"3770bf76b89b9ecb603a450a140b498c5ab210a702c8dafc26fe9106dd0c3269"}
Sep 10 16:17:12.164767 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:12.164716    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-40.ec2.internal" podStartSLOduration=3.164698435 podStartE2EDuration="3.164698435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:12.163846776 +0000 UTC m=+4.565647903" watchObservedRunningTime="2026-09-10 16:17:12.164698435 +0000 UTC m=+4.566499562"
Sep 10 16:17:13.108587 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:13.108505    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:13.109138 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.108641    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:13.178238 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:13.178204    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" event={"ID":"4e548e3497cb82467734f20174cf45c0","Type":"ContainerStarted","Data":"300dfe817258b6a981a1edafc3f4439013d03193a3f0586dbe74b62d449816de"}
Sep 10 16:17:13.650908 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:13.650869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:13.651078 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:13.650929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:13.651157 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651073    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:13.651157 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651093    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:13.651157 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651108    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:13.651316 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651168    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:17.651148538 +0000 UTC m=+10.052949657 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:13.651600 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651574    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:13.651715 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.651630    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:17.651616373 +0000 UTC m=+10.053417479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:13.752135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:13.752097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:13.752438 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.752418    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:13.752527 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:13.752497    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:17.752478914 +0000 UTC m=+10.154280036 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:14.108149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:14.108072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:14.108324 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:14.108236    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:14.108491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:14.108461    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:14.108618 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:14.108590    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:15.107028 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:15.106899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:15.107223 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:15.107037    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:16.107967 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:16.107873    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:16.108440 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:16.108034    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:16.108440 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:16.108088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:16.108440 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:16.108184    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:17.106963 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:17.106877    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:17.107113 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.107014    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:17.686791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:17.686853    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.686896    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.686959    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:25.686942325 +0000 UTC m=+18.088743453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.686984    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.686999    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.687012    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:17.687076 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.687049    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:25.687038458 +0000 UTC m=+18.088839563 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:17.787698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:17.787649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:17.787900 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.787805    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:17.787900 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:17.787872    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:25.787853941 +0000 UTC m=+18.189655060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:18.108360 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:18.107685    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:18.108360 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:18.107799    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:18.108360 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:18.108146    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:18.108360 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:18.108249    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:19.106543 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:19.106056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:19.106543 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:19.106186    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:20.106387 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:20.106351    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:20.106575 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:20.106498    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:20.106575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:20.106541    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:20.106966 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:20.106620    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:21.106781 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:21.106744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:21.107169 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:21.106880    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:22.106519 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:22.106484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:22.106710 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:22.106633    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:22.106710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:22.106698    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:22.106843 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:22.106823    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:23.106400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:23.106366    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:23.106569 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:23.106488    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:24.106964 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:24.106900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:24.108399 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:24.108363    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:24.108399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:24.108377    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:24.108597 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:24.108526    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:25.106822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:25.106778    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:25.107044 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.106926    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:25.750596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:25.750693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750778    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750783    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750810    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750825    2577 projected.go:196] Error preparing data for projected volume kube-api-access-t9bml for pod openshift-network-diagnostics/network-check-target-sdlsb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:25.750837 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750833    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:17:41.750814747 +0000 UTC m=+34.152615867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:25.751289 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.750874    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml podName:114fad46-62e3-4b42-97c0-643871c1b119 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:41.750857384 +0000 UTC m=+34.152658528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-t9bml" (UniqueName: "kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml") pod "network-check-target-sdlsb" (UID: "114fad46-62e3-4b42-97c0-643871c1b119") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 16:17:25.851126 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:25.851091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:25.851298 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.851220    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:25.851298 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:25.851297    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:41.851265455 +0000 UTC m=+34.253066560 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:26.106039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:26.105961    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:26.106200 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:26.106102    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:26.106200 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:26.106162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:26.106326 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:26.106260    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:27.106413 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:27.106376    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:27.106897 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:27.106502    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:28.107456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:28.106916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:28.107456 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:28.107090    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:28.107456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:28.107175    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:28.107456 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:28.107289    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:29.106305 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.105934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:29.106516 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:29.106432    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:29.211926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.211898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"d70069349e1f50576d4e3bf21514bc13e111eab839aff59730a352ce215e0ada"}
Sep 10 16:17:29.212494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.211987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"e387583860621b6d480ca5c376d87335dd16866d069a4ff34fee482322982bcb"}
Sep 10 16:17:29.212494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.212041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"b7061af2afccab9d65530a88e152af5cd043e63d9d7854d20736460a454b550d"}
Sep 10 16:17:29.212494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.212089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"62b9a7d5f83dcec04817aeec26c6fc511759f2fcd99697607ac27077362db585"}
Sep 10 16:17:29.212494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.212130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"797d59d4539af577501f2efdf7ed0eb3c5df7fa131d0951d065a056eede69f5b"}
Sep 10 16:17:29.213200 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.213181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-r84rk" event={"ID":"9ab87d95-1588-4a35-9211-d0d53b937dd2","Type":"ContainerStarted","Data":"93a344155a28529403c177a4af311cefd9fc356a06787146b8e75941ad385e35"}
Sep 10 16:17:29.214662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.214642    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-95whm" event={"ID":"dc1d39a1-3c08-48dd-84b3-ff9ebbf63831","Type":"ContainerStarted","Data":"29b389b6399355542b01867b916f92c3d0d8417ce91ee2456212833ee6515832"}
Sep 10 16:17:29.216188 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.216168    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="cd08d7914a335648afde39baaae2b889b6ac25996e8bb0bfe69754dcd7a547b9" exitCode=0
Sep 10 16:17:29.216295 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.216194    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"cd08d7914a335648afde39baaae2b889b6ac25996e8bb0bfe69754dcd7a547b9"}
Sep 10 16:17:29.218861 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.218836    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d24lj" event={"ID":"1500121e-5856-4e8a-96a1-a7815dc5905c","Type":"ContainerStarted","Data":"e1efec3457b8d91dd9634a00c27eaf6f474d338f9263b08ca88305c6bd10e7a1"}
Sep 10 16:17:29.221172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.221147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9jvcb" event={"ID":"ac6a08bb-26c2-4a3a-a9c4-1ca373eafc53","Type":"ContainerStarted","Data":"443df0e02688179f4eafdac5ff801306d097df8e2b7f236cf5f128a46f6fb4f0"}
Sep 10 16:17:29.222720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.222699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w27lk" event={"ID":"72045530-1543-405a-9763-852fb5e9c22f","Type":"ContainerStarted","Data":"7bf58d05d934e4f1f4cc271d8e5c77790c59bc3bdf513dbf0d913621106f052f"}
Sep 10 16:17:29.224116 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.224099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4" event={"ID":"6b7858a1-8d78-4152-8162-0555789ff294","Type":"ContainerStarted","Data":"a32d33070450e0d09f7b8f65e627c03a9d6dab9452891dd71b153fef1aac3f7e"}
Sep 10 16:17:29.232195 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.232155    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-r84rk" podStartSLOduration=3.043371497 podStartE2EDuration="21.232142957s" totalImagesPullingTime="18.18877146s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:29.231465624 +0000 UTC m=+21.633266755" watchObservedRunningTime="2026-09-10 16:17:29.232142957 +0000 UTC m=+21.633944085"
Sep 10 16:17:29.232332 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.232309    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-40.ec2.internal" podStartSLOduration=20.232300388 podStartE2EDuration="20.232300388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:13.198158309 +0000 UTC m=+5.599959437" watchObservedRunningTime="2026-09-10 16:17:29.232300388 +0000 UTC m=+21.634101507"
Sep 10 16:17:29.248351 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.248307    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d24lj" podStartSLOduration=3.015276802 podStartE2EDuration="21.248270566s" totalImagesPullingTime="18.232993764s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:29.247946376 +0000 UTC m=+21.649747505" watchObservedRunningTime="2026-09-10 16:17:29.248270566 +0000 UTC m=+21.650071747"
Sep 10 16:17:29.268971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.268922    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9jvcb" podStartSLOduration=2.973141977 podStartE2EDuration="21.26890185s" totalImagesPullingTime="18.295759873s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:29.268261423 +0000 UTC m=+21.670062552" watchObservedRunningTime="2026-09-10 16:17:29.26890185 +0000 UTC m=+21.670703018"
Sep 10 16:17:29.342818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.342728    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w27lk" podStartSLOduration=10.381708153 podStartE2EDuration="21.342713002s" totalImagesPullingTime="10.961004849s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:29.342513739 +0000 UTC m=+21.744314857" watchObservedRunningTime="2026-09-10 16:17:29.342713002 +0000 UTC m=+21.744514129"
Sep 10 16:17:29.343160 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:29.343139    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-95whm" podStartSLOduration=2.885422536 podStartE2EDuration="21.343134499s" totalImagesPullingTime="18.457711963s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:29.320198619 +0000 UTC m=+21.721999747" watchObservedRunningTime="2026-09-10 16:17:29.343134499 +0000 UTC m=+21.744935622"
Sep 10 16:17:30.106114 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.106083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:30.106270 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:30.106173    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:30.106326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.106266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:30.106377 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:30.106358    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:30.221139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.221111    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 16:17:30.229111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.229085    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4" event={"ID":"6b7858a1-8d78-4152-8162-0555789ff294","Type":"ContainerStarted","Data":"d7a355f0bd7925ef024a43f5ca2ab86b7b58f4c7dfa40153f0238af0a824b171"}
Sep 10 16:17:30.232505 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.232479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"81ad7f95ee586bdd5be0c7f8a31a075b622a656f43fcc4549896118ec7b15382"}
Sep 10 16:17:30.234093 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.234069    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4r9t" event={"ID":"9780784f-b25a-4cb6-9ee6-54cb59bf3a60","Type":"ContainerStarted","Data":"eabcf3ddecd10d12c37b75466b4d4abaf2c698a6fe92d144ddb6fd117e1869ec"}
Sep 10 16:17:30.254163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:30.254083    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4r9t" podStartSLOduration=3.982530821 podStartE2EDuration="22.254069875s" totalImagesPullingTime="18.271539054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:30.253831708 +0000 UTC m=+22.655632834" watchObservedRunningTime="2026-09-10 16:17:30.254069875 +0000 UTC m=+22.655871002"
Sep 10 16:17:31.106318 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:31.106271    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:31.106506 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:31.106418    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:31.217209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:31.217063    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T16:17:30.221134702Z","UUID":"9196c576-5968-46b5-98e2-a10232ee9013","Handler":null,"Name":"","Endpoint":""}
Sep 10 16:17:31.219253 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:31.219212    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 16:17:31.219253 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:31.219249    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 16:17:32.107057 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:32.106979    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:32.107470 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:32.106988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:32.107470 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:32.107126    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:32.107561 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:32.107534    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:32.240677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:32.240633    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4" event={"ID":"6b7858a1-8d78-4152-8162-0555789ff294","Type":"ContainerStarted","Data":"9dc22168f8709a1946f176e3eaa140a209a3c3709341618d3eaddf962bcebee1"}
Sep 10 16:17:32.243753 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:32.243721    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"696ffc4ade2c54f00f559e66b549695df77847871ed381ea9fd2a38ba66eccca"}
Sep 10 16:17:32.264218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:32.264166    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-djbs4" podStartSLOduration=3.431265315 podStartE2EDuration="24.264150733s" totalImagesPullingTime="20.832885418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:32.263716148 +0000 UTC m=+24.665517275" watchObservedRunningTime="2026-09-10 16:17:32.264150733 +0000 UTC m=+24.665951860"
Sep 10 16:17:33.105984 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:33.105947    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:33.106174 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:33.106089    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:33.263914 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:33.263872    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:33.264705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:33.264681    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:34.106579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:34.106549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:34.106741 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:34.106662    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:34.106967 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:34.106555    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:34.107109 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:34.107083    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:34.249117 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:34.249090    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:34.249660 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:34.249641    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w27lk"
Sep 10 16:17:35.106589 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.106425    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:35.106934 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:35.106665    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:35.252752 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.252720    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="52dda39839c9f1c79b2b7aec42a2f12c0fed674573c3d7dcc1238fdd8a3108bc" exitCode=0
Sep 10 16:17:35.252907 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.252803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"52dda39839c9f1c79b2b7aec42a2f12c0fed674573c3d7dcc1238fdd8a3108bc"}
Sep 10 16:17:35.256882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.256858    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" event={"ID":"89b9e2a7-13a7-40eb-9194-06c2f9bad9a2","Type":"ContainerStarted","Data":"780d2deb705d3b2328e54cad2e30b34c41602d294285c836d132520c6a819de0"}
Sep 10 16:17:35.257111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.257093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:35.257174 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.257161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:35.272788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.272759    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:35.307060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:35.307013    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-n746n" podStartSLOduration=8.954207101 podStartE2EDuration="27.307000729s" totalImagesPullingTime="18.352793628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:35.306210383 +0000 UTC m=+27.708011532" watchObservedRunningTime="2026-09-10 16:17:35.307000729 +0000 UTC m=+27.708801856"
Sep 10 16:17:36.106971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.106944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:36.107589 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.106950    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:36.107589 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:36.107033    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:36.107589 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:36.107143    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:36.261496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.261468    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="26dc6168b913873938e7a176ea82dbbe21082147744aa1ed26747a8b8d91f824" exitCode=0
Sep 10 16:17:36.261604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.261546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"26dc6168b913873938e7a176ea82dbbe21082147744aa1ed26747a8b8d91f824"}
Sep 10 16:17:36.262028 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.262007    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:36.276063 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.276044    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:17:36.428195 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.428169    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xwrj7"]
Sep 10 16:17:36.428394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.428290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:36.428394 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:36.428374    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:36.431353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.431329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdlsb"]
Sep 10 16:17:36.431471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.431408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:36.431520 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:36.431489    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:36.434208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.434181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87mpg"]
Sep 10 16:17:36.434337 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:36.434290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:36.434404 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:36.434387    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:37.265771 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:37.265741    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="ba25e44a793ae74fae3a108bd790e66fdff83f94e741f45542b876c4500280fe" exitCode=0
Sep 10 16:17:37.266322 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:37.265829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"ba25e44a793ae74fae3a108bd790e66fdff83f94e741f45542b876c4500280fe"}
Sep 10 16:17:38.107757 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:38.107562    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:38.107924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:38.107621    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:38.107924 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:38.107836    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:38.107924 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:38.107886    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:38.107924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:38.107639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:38.108150 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:38.107988    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:40.106701 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:40.106667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:40.107135 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:40.106793    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-sdlsb" podUID="114fad46-62e3-4b42-97c0-643871c1b119"
Sep 10 16:17:40.107135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:40.106967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:40.107135 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:40.107068    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-87mpg" podUID="13542d90-fc55-4433-aeb2-8ebf3cc57ee0"
Sep 10 16:17:40.107135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:40.107121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:40.107366 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:40.107238    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xwrj7" podUID="0bdb4c52-c0b9-4029-a2b2-ed39461af0ce"
Sep 10 16:17:41.392221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.392186    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-40.ec2.internal" event="NodeReady"
Sep 10 16:17:41.392710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.392365    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 16:17:41.431383 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.431344    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"]
Sep 10 16:17:41.460289 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.460246    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"]
Sep 10 16:17:41.460488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.460396    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.466562 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.466536    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 10 16:17:41.466562 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.466536    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.466734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.466636    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.466917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.466904    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-qqmrj\""
Sep 10 16:17:41.467202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.467184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 10 16:17:41.478299 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.478254    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"]
Sep 10 16:17:41.478469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.478428    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.483491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.483461    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 10 16:17:41.483630 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.483541    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.483788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.483766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9j87p\""
Sep 10 16:17:41.483876 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.483845    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 10 16:17:41.484196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.484039    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.498491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.498432    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"]
Sep 10 16:17:41.498611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.498561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"
Sep 10 16:17:41.501904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.501834    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.502291 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.502256    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.502521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.502500    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-56qzp\""
Sep 10 16:17:41.518450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.518425    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:17:41.518568 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.518551    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.521420 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.521396    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 10 16:17:41.521576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.521554    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.521662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.521554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-jfmcw\""
Sep 10 16:17:41.521722 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.521677    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 10 16:17:41.521777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.521678    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.539350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.539324    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-59648f9df8-wbvrp"]
Sep 10 16:17:41.539805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.539788    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.542817 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.542793    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 16:17:41.542919 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.542823    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 16:17:41.542919 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.542857    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s2v9s\""
Sep 10 16:17:41.543034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.542949    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 16:17:41.549157 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.549138    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 16:17:41.563195 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-config\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.563362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.563362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-serving-cert\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.563362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lqls\" (UniqueName: \"kubernetes.io/projected/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-kube-api-access-4lqls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.563362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g8fs\" (UniqueName: \"kubernetes.io/projected/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-kube-api-access-5g8fs\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.563551 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.563383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.565997 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.565978    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"]
Sep 10 16:17:41.566139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.566119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.569096 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569074    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 10 16:17:41.569214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569129    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 10 16:17:41.569214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569140    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 10 16:17:41.569366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 10 16:17:41.569366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569334    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.569455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569413    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9sw7m\""
Sep 10 16:17:41.569558 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.569540    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.579464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.579440    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"]
Sep 10 16:17:41.579559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.579484    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"]
Sep 10 16:17:41.579559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.579506    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"]
Sep 10 16:17:41.579559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.579527    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-tm6h9"]
Sep 10 16:17:41.579674 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.579561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.582529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.582512    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.582630 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.582574    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rx6s8\""
Sep 10 16:17:41.582731 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.582714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 10 16:17:41.582886 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.582870    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.595617 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.595585    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ctsf5"]
Sep 10 16:17:41.595820 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.595800    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.598450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.598427    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 16:17:41.598579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.598561    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 16:17:41.598692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.598672    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wwcbb\""
Sep 10 16:17:41.611909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.611884    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xm69q"]
Sep 10 16:17:41.612031 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.612007    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.615019 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.614996    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.615143 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.615108    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 10 16:17:41.615204 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.614998    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.615400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.615352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-mj55p\""
Sep 10 16:17:41.615400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.615352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 10 16:17:41.620566 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.620544    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 10 16:17:41.630559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.630538    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mqnrc"]
Sep 10 16:17:41.630671 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.630644    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.633801 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.633779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 16:17:41.633922 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.633861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.633922 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.633861    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.633922 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.633883    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kr9bx\""
Sep 10 16:17:41.645452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.645419    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w27lk_72045530-1543-405a-9763-852fb5e9c22f/konnectivity-agent/0.log" limit=1024
Sep 10 16:17:41.654621 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.654602    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cpn8t"]
Sep 10 16:17:41.654747 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.654732    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc"
Sep 10 16:17:41.659168 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.659151    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.659267 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.659211    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-tdhdx\""
Sep 10 16:17:41.659355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.659212    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.663969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.663945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0d2e7938-d60f-4b3f-9d05-562219253f14-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.664067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.663981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-tmp-dir\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.664172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shj6p\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664130    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.664172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q8r4\" (UniqueName: \"kubernetes.io/projected/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-kube-api-access-2q8r4\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664171    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664191    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw5l5\" (UniqueName: \"kubernetes.io/projected/b8629aee-eb84-4e99-8b93-259f97a3dbd0-kube-api-access-bw5l5\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-default-certificate\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-config\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnz2f\" (UniqueName: \"kubernetes.io/projected/0094cd2c-52d0-4501-be7d-cb3b6b13b573-kube-api-access-wnz2f\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.664394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664492    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-serving-cert\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664521    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-config-volume\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g8fs\" (UniqueName: \"kubernetes.io/projected/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-kube-api-access-5g8fs\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664643    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.664675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-stats-auth\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.664966 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.664966 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.664966 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-config\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.664966 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0d2e7938-d60f-4b3f-9d05-562219253f14-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.664966 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lqls\" (UniqueName: \"kubernetes.io/projected/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-kube-api-access-4lqls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.664969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.665004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skgll\" (UniqueName: \"kubernetes.io/projected/29da8dc0-47a0-4dfa-ac9e-82807f94fefc-kube-api-access-skgll\") pod \"volume-data-source-validator-695c895bd6-fd8s7\" (UID: \"29da8dc0-47a0-4dfa-ac9e-82807f94fefc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.665039    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.665055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2p7l\" (UniqueName: \"kubernetes.io/projected/0d2e7938-d60f-4b3f-9d05-562219253f14-kube-api-access-p2p7l\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.665152    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:41.665208 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.665202    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.165189424 +0000 UTC m=+34.566990532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:41.665608 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.665566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.668863 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.668825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-serving-cert\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.670265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670156    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:17:41.670265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mqnrc"]
Sep 10 16:17:41.670265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670215    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"]
Sep 10 16:17:41.670265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670234    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59648f9df8-wbvrp"]
Sep 10 16:17:41.670265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.670613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ctsf5"]
Sep 10 16:17:41.670613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670513    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm6h9"]
Sep 10 16:17:41.670613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xm69q"]
Sep 10 16:17:41.670613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670562    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"]
Sep 10 16:17:41.670613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.670587    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cpn8t"]
Sep 10 16:17:41.673304 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.673262    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 16:17:41.673404 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.673330    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 16:17:41.674329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.673976    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:41.674329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.674016    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 16:17:41.674512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.674493    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ggrtd\""
Sep 10 16:17:41.686768 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.686747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lqls\" (UniqueName: \"kubernetes.io/projected/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-kube-api-access-4lqls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:41.686869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.686751    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g8fs\" (UniqueName: \"kubernetes.io/projected/58a190c9-70a4-4c84-96c7-e9c6f4b2214f-kube-api-access-5g8fs\") pod \"service-ca-operator-859f94c4b-2j8lp\" (UID: \"58a190c9-70a4-4c84-96c7-e9c6f4b2214f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.688735 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.688710    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 16:17:41.754954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.754855    2577 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-w27lk_72045530-1543-405a-9763-852fb5e9c22f/konnectivity-agent/0.log" limit=1024
Sep 10 16:17:41.765852 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765825    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.765991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765866    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-snapshots\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.765991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.765991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-serving-cert\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.765991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765945    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.765991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.765974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-config-volume\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.766019    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766031    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lq9z\" (UniqueName: \"kubernetes.io/projected/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-kube-api-access-5lq9z\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.766089    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.266067269 +0000 UTC m=+34.667868378 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-stats-auth\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766169    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.766213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766187    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0d2e7938-d60f-4b3f-9d05-562219253f14-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.766495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bd792\" (UniqueName: \"kubernetes.io/projected/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-kube-api-access-bd792\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.766495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.766495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skgll\" (UniqueName: \"kubernetes.io/projected/29da8dc0-47a0-4dfa-ac9e-82807f94fefc-kube-api-access-skgll\") pod \"volume-data-source-validator-695c895bd6-fd8s7\" (UID: \"29da8dc0-47a0-4dfa-ac9e-82807f94fefc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"
Sep 10 16:17:41.766647 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-config-volume\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.766711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766633    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.766887 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.766867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0d2e7938-d60f-4b3f-9d05-562219253f14-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.767142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:41.767269 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767164    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2p7l\" (UniqueName: \"kubernetes.io/projected/0d2e7938-d60f-4b3f-9d05-562219253f14-kube-api-access-p2p7l\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.767269 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.767191    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:41.767269 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0d2e7938-d60f-4b3f-9d05-562219253f14-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.767269 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.767253    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.267230511 +0000 UTC m=+34.669031633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767296    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-tmp-dir\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.767354    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret podName:0bdb4c52-c0b9-4029-a2b2-ed39461af0ce nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.767331971 +0000 UTC m=+66.169133090 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret") pod "global-pull-secret-syncer-xwrj7" (UID: "0bdb4c52-c0b9-4029-a2b2-ed39461af0ce") : object "kube-system"/"original-pull-secret" not registered
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shj6p\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.767515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5thj\" (UniqueName: \"kubernetes.io/projected/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-kube-api-access-c5thj\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q8r4\" (UniqueName: \"kubernetes.io/projected/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-kube-api-access-2q8r4\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-tmp-dir\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.767648    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.767689    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.267675643 +0000 UTC m=+34.669476762 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767717    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzxmc\" (UniqueName: \"kubernetes.io/projected/4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d-kube-api-access-pzxmc\") pod \"network-check-source-685456655f-mqnrc\" (UID: \"4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767813    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-tmp\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw5l5\" (UniqueName: \"kubernetes.io/projected/b8629aee-eb84-4e99-8b93-259f97a3dbd0-kube-api-access-bw5l5\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.767896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-serving-cert\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-trusted-ca\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767957    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-default-certificate\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.767982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-config\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.768028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-cert\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.768061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.768078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.768086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnz2f\" (UniqueName: \"kubernetes.io/projected/0094cd2c-52d0-4501-be7d-cb3b6b13b573-kube-api-access-wnz2f\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.768338    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.768397    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.268379103 +0000 UTC m=+34.670180231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.768481    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:41.768489 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.768492    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:41.769396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.769165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.769591 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.768921    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:42.268902486 +0000 UTC m=+34.670703594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:41.769800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.769769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.770371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.770344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-stats-auth\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.770371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.770348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9bml\" (UniqueName: \"kubernetes.io/projected/114fad46-62e3-4b42-97c0-643871c1b119-kube-api-access-t9bml\") pod \"network-check-target-sdlsb\" (UID: \"114fad46-62e3-4b42-97c0-643871c1b119\") " pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:41.770596 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.770575    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0d2e7938-d60f-4b3f-9d05-562219253f14-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.773358 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.772209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-default-certificate\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.773358 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.773291    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.774983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.774964    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"
Sep 10 16:17:41.781486 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.781440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnz2f\" (UniqueName: \"kubernetes.io/projected/0094cd2c-52d0-4501-be7d-cb3b6b13b573-kube-api-access-wnz2f\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:41.781719 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.781699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shj6p\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.782094 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.782048    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2p7l\" (UniqueName: \"kubernetes.io/projected/0d2e7938-d60f-4b3f-9d05-562219253f14-kube-api-access-p2p7l\") pod \"kube-storage-version-migrator-operator-8d7dd4675-hln7r\" (UID: \"0d2e7938-d60f-4b3f-9d05-562219253f14\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.783407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.783383    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:41.783707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.783659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skgll\" (UniqueName: \"kubernetes.io/projected/29da8dc0-47a0-4dfa-ac9e-82807f94fefc-kube-api-access-skgll\") pod \"volume-data-source-validator-695c895bd6-fd8s7\" (UID: \"29da8dc0-47a0-4dfa-ac9e-82807f94fefc\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"
Sep 10 16:17:41.784635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.784613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw5l5\" (UniqueName: \"kubernetes.io/projected/b8629aee-eb84-4e99-8b93-259f97a3dbd0-kube-api-access-bw5l5\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:41.784702 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.784691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q8r4\" (UniqueName: \"kubernetes.io/projected/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-kube-api-access-2q8r4\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:41.809131 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.809110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"
Sep 10 16:17:41.828355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.828330    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"
Sep 10 16:17:41.869503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:41.869503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bd792\" (UniqueName: \"kubernetes.io/projected/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-kube-api-access-bd792\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.869684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5thj\" (UniqueName: \"kubernetes.io/projected/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-kube-api-access-c5thj\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.869684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.869684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzxmc\" (UniqueName: \"kubernetes.io/projected/4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d-kube-api-access-pzxmc\") pod \"network-check-source-685456655f-mqnrc\" (UID: \"4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc"
Sep 10 16:17:41.869684 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.869632    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:41.869684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869665    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-tmp\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:41.869700    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs podName:13542d90-fc55-4433-aeb2-8ebf3cc57ee0 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.869680573 +0000 UTC m=+66.271481693 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs") pod "network-metrics-daemon-87mpg" (UID: "13542d90-fc55-4433-aeb2-8ebf3cc57ee0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-serving-cert\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869758    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-trusted-ca\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869791    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-config\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-cert\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-snapshots\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.869937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869922    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-serving-cert\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.870209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.870209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.869990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lq9z\" (UniqueName: \"kubernetes.io/projected/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-kube-api-access-5lq9z\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.870209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.870010    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-tmp\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.870209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.870173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.870515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.870491    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-snapshots\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.870896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.870850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.871000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.870917    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-config\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.871063 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.871003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-trusted-ca\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.872638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.872618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-serving-cert\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.872735 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.872664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-serving-cert\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.872883 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.872866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-cert\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.879129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.879105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lq9z\" (UniqueName: \"kubernetes.io/projected/71fd4ecd-9ead-4a5f-8827-82fbd67d4812-kube-api-access-5lq9z\") pod \"insights-operator-76bcd6ccf5-ctsf5\" (UID: \"71fd4ecd-9ead-4a5f-8827-82fbd67d4812\") " pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.880335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.880312    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzxmc\" (UniqueName: \"kubernetes.io/projected/4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d-kube-api-access-pzxmc\") pod \"network-check-source-685456655f-mqnrc\" (UID: \"4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d\") " pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc"
Sep 10 16:17:41.880430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.880334    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bd792\" (UniqueName: \"kubernetes.io/projected/a7d7b92b-7659-4bf4-a95a-4d5364e3578a-kube-api-access-bd792\") pod \"console-operator-85b454874b-cpn8t\" (UID: \"a7d7b92b-7659-4bf4-a95a-4d5364e3578a\") " pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:41.880584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.880568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5thj\" (UniqueName: \"kubernetes.io/projected/9fc228f9-6b63-4f2b-aca1-c4720725d2d0-kube-api-access-c5thj\") pod \"ingress-canary-xm69q\" (UID: \"9fc228f9-6b63-4f2b-aca1-c4720725d2d0\") " pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.922808 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.922771    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5"
Sep 10 16:17:41.939958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.939929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xm69q"
Sep 10 16:17:41.964328 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.964303    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc"
Sep 10 16:17:41.984542 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:41.984513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:17:42.106625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.106540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:42.106625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.106559    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:17:42.106834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.106540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:17:42.110104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.110082    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 16:17:42.110228 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.110134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-htvxx\""
Sep 10 16:17:42.110336 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.110083    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j8r7d\""
Sep 10 16:17:42.110411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.110338    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 16:17:42.117136 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.117120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:42.172775 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.172658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:42.172951 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.172843    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:42.172951 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.172934    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.172910319 +0000 UTC m=+35.574711429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:42.273445 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.273404    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:42.273626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.273460    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:42.273626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.273523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:42.273626 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273577    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:42.273626 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273601    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:42.273626 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273608    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.273632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273661    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.273641982 +0000 UTC m=+35.675443092 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273684    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.273675157 +0000 UTC m=+35.675476263 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273713    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.273729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273763    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.273746337 +0000 UTC m=+35.675547445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273792    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273824    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.273814167 +0000 UTC m=+35.675615272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:42.273896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:42.273861    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:43.273849847 +0000 UTC m=+35.675650996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:42.915455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.915263    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7"]
Sep 10 16:17:42.916415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.916393    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp"]
Sep 10 16:17:42.919886 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.918418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cpn8t"]
Sep 10 16:17:42.919886 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.919521    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-ctsf5"]
Sep 10 16:17:42.920252 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.920058    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58a190c9_70a4_4c84_96c7_e9c6f4b2214f.slice/crio-50673193ac3726f725054bb54848bd5bebb759eb72583a463a0598f736d7d5db WatchSource:0}: Error finding container 50673193ac3726f725054bb54848bd5bebb759eb72583a463a0598f736d7d5db: Status 404 returned error can't find the container with id 50673193ac3726f725054bb54848bd5bebb759eb72583a463a0598f736d7d5db
Sep 10 16:17:42.920429 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.920404    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29da8dc0_47a0_4dfa_ac9e_82807f94fefc.slice/crio-a9325b0deaea6f922b50410c7b79f4531f0789ba41e4256b8ba8bf43af0a2f9d WatchSource:0}: Error finding container a9325b0deaea6f922b50410c7b79f4531f0789ba41e4256b8ba8bf43af0a2f9d: Status 404 returned error can't find the container with id a9325b0deaea6f922b50410c7b79f4531f0789ba41e4256b8ba8bf43af0a2f9d
Sep 10 16:17:42.921728 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.921702    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7d7b92b_7659_4bf4_a95a_4d5364e3578a.slice/crio-3889b545f48d2c5d79e37fcb32e305558fa974f3a7a4f321f38284ce307924c2 WatchSource:0}: Error finding container 3889b545f48d2c5d79e37fcb32e305558fa974f3a7a4f321f38284ce307924c2: Status 404 returned error can't find the container with id 3889b545f48d2c5d79e37fcb32e305558fa974f3a7a4f321f38284ce307924c2
Sep 10 16:17:42.922725 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.922691    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fd4ecd_9ead_4a5f_8827_82fbd67d4812.slice/crio-ee306a5a5096e4e668c900194e3095c03a26d243de43a1c9668c7cdfec08d063 WatchSource:0}: Error finding container ee306a5a5096e4e668c900194e3095c03a26d243de43a1c9668c7cdfec08d063: Status 404 returned error can't find the container with id ee306a5a5096e4e668c900194e3095c03a26d243de43a1c9668c7cdfec08d063
Sep 10 16:17:42.929464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.929439    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r"]
Sep 10 16:17:42.933607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.933581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-mqnrc"]
Sep 10 16:17:42.933718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.933639    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xm69q"]
Sep 10 16:17:42.934067 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.934044    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2e7938_d60f_4b3f_9d05_562219253f14.slice/crio-426b9956a3f914e12b72cd57e393ec6946b4a130c8054ca645d3a792133acb86 WatchSource:0}: Error finding container 426b9956a3f914e12b72cd57e393ec6946b4a130c8054ca645d3a792133acb86: Status 404 returned error can't find the container with id 426b9956a3f914e12b72cd57e393ec6946b4a130c8054ca645d3a792133acb86
Sep 10 16:17:42.938333 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.938001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f55c1a5_ceb3_4cf2_90d6_55dbd9a1627d.slice/crio-087a7018491722f8d05743b40b9c1305bfe07b6c91f304710d53888b61bbdb38 WatchSource:0}: Error finding container 087a7018491722f8d05743b40b9c1305bfe07b6c91f304710d53888b61bbdb38: Status 404 returned error can't find the container with id 087a7018491722f8d05743b40b9c1305bfe07b6c91f304710d53888b61bbdb38
Sep 10 16:17:42.939736 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.939360    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fc228f9_6b63_4f2b_aca1_c4720725d2d0.slice/crio-57656fe3d3c5f07501de005bad3d602f54ba3a612000517e899c8115957e3288 WatchSource:0}: Error finding container 57656fe3d3c5f07501de005bad3d602f54ba3a612000517e899c8115957e3288: Status 404 returned error can't find the container with id 57656fe3d3c5f07501de005bad3d602f54ba3a612000517e899c8115957e3288
Sep 10 16:17:42.945164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:42.945126    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-sdlsb"]
Sep 10 16:17:42.948525 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:42.948505    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114fad46_62e3_4b42_97c0_643871c1b119.slice/crio-d9b421d139b10940013ab1a5cd8151f7fd7ebd83856d98331f9d5829d0b0918d WatchSource:0}: Error finding container d9b421d139b10940013ab1a5cd8151f7fd7ebd83856d98331f9d5829d0b0918d: Status 404 returned error can't find the container with id d9b421d139b10940013ab1a5cd8151f7fd7ebd83856d98331f9d5829d0b0918d
Sep 10 16:17:43.181980 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.181953    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:43.182125 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.182105    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:43.182183 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.182174    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.182158503 +0000 UTC m=+37.583959608 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:43.282386 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.282353    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:43.282541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.282417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:43.282541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.282506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:43.282541 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282513    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.282559    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282564    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.282541041 +0000 UTC m=+37.684342167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282597    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.282581426 +0000 UTC m=+37.684382535 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282597    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282649    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282674    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.282629705 +0000 UTC m=+37.684430811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282693    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.282681575 +0000 UTC m=+37.684482713 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:43.282714 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.282714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:43.283031 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282806    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:43.283031 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282820    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:43.283031 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:43.282851    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:45.282843631 +0000 UTC m=+37.684644737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:43.283133 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.283050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp" event={"ID":"58a190c9-70a4-4c84-96c7-e9c6f4b2214f","Type":"ContainerStarted","Data":"50673193ac3726f725054bb54848bd5bebb759eb72583a463a0598f736d7d5db"}
Sep 10 16:17:43.285748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.285726    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="ef1db949eb66bcd5e2653af1e2058b37f0bc6c5f8fd46b92d7319f99ecf4e74f" exitCode=0
Sep 10 16:17:43.285841 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.285794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"ef1db949eb66bcd5e2653af1e2058b37f0bc6c5f8fd46b92d7319f99ecf4e74f"}
Sep 10 16:17:43.286919 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.286901    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xm69q" event={"ID":"9fc228f9-6b63-4f2b-aca1-c4720725d2d0","Type":"ContainerStarted","Data":"57656fe3d3c5f07501de005bad3d602f54ba3a612000517e899c8115957e3288"}
Sep 10 16:17:43.288130 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.288111    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc" event={"ID":"4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d","Type":"ContainerStarted","Data":"087a7018491722f8d05743b40b9c1305bfe07b6c91f304710d53888b61bbdb38"}
Sep 10 16:17:43.289206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.289174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r" event={"ID":"0d2e7938-d60f-4b3f-9d05-562219253f14","Type":"ContainerStarted","Data":"426b9956a3f914e12b72cd57e393ec6946b4a130c8054ca645d3a792133acb86"}
Sep 10 16:17:43.290433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.290374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7" event={"ID":"29da8dc0-47a0-4dfa-ac9e-82807f94fefc","Type":"ContainerStarted","Data":"a9325b0deaea6f922b50410c7b79f4531f0789ba41e4256b8ba8bf43af0a2f9d"}
Sep 10 16:17:43.291495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.291476    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" event={"ID":"a7d7b92b-7659-4bf4-a95a-4d5364e3578a","Type":"ContainerStarted","Data":"3889b545f48d2c5d79e37fcb32e305558fa974f3a7a4f321f38284ce307924c2"}
Sep 10 16:17:43.292535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.292519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5" event={"ID":"71fd4ecd-9ead-4a5f-8827-82fbd67d4812","Type":"ContainerStarted","Data":"ee306a5a5096e4e668c900194e3095c03a26d243de43a1c9668c7cdfec08d063"}
Sep 10 16:17:43.293553 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:43.293537    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdlsb" event={"ID":"114fad46-62e3-4b42-97c0-643871c1b119","Type":"ContainerStarted","Data":"d9b421d139b10940013ab1a5cd8151f7fd7ebd83856d98331f9d5829d0b0918d"}
Sep 10 16:17:44.311593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:44.310524    2577 generic.go:356] "Generic (PLEG): container finished" podID="534d15f4-9147-439b-8a20-9d951e5cf6f3" containerID="d3003496f0fbdf155948bf42a7b7379a6f9e643a8a4999c9331c7b552cb15c90" exitCode=0
Sep 10 16:17:44.311593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:44.310645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerDied","Data":"d3003496f0fbdf155948bf42a7b7379a6f9e643a8a4999c9331c7b552cb15c90"}
Sep 10 16:17:45.207519 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.206811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:45.207519 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.207043    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:45.207519 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.207114    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.207093207 +0000 UTC m=+41.608894317 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.307417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.307487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.307539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.307577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.307623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.307841    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.307821183 +0000 UTC m=+41.709622309 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308298    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308350    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.308334336 +0000 UTC m=+41.710135443 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308404    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308437    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.308427351 +0000 UTC m=+41.710228457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308496    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308506    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308534    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.308524417 +0000 UTC m=+41.710325525 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308582    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:45.308662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:45.308610    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:49.308600566 +0000 UTC m=+41.710401675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:45.319333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.319295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7" event={"ID":"29da8dc0-47a0-4dfa-ac9e-82807f94fefc","Type":"ContainerStarted","Data":"76686c3501766bb2ce69d4174f500bd7ff36de20a1f9d5406e59668e4ae3cba6"}
Sep 10 16:17:45.326895 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.326250    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" event={"ID":"534d15f4-9147-439b-8a20-9d951e5cf6f3","Type":"ContainerStarted","Data":"7d5844c8de1fdc9e0ece3855c12caab6c53d056260e1a469d674f189cbe8f598"}
Sep 10 16:17:45.340284 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.340082    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-fd8s7" podStartSLOduration=8.185058263 podStartE2EDuration="10.340059652s" totalImagesPullingTime="2.155001389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:45.33984516 +0000 UTC m=+37.741646287" watchObservedRunningTime="2026-09-10 16:17:45.340059652 +0000 UTC m=+37.741860779"
Sep 10 16:17:45.367377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:45.366852    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hr6l7" podStartSLOduration=6.533211939 podStartE2EDuration="37.366829468s" totalImagesPullingTime="30.833617529s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:45.364107965 +0000 UTC m=+37.765909132" watchObservedRunningTime="2026-09-10 16:17:45.366829468 +0000 UTC m=+37.768630597"
Sep 10 16:17:49.246890 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.246845    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:49.247417 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.247018    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:49.247417 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.247094    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.247076943 +0000 UTC m=+49.648878064 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:49.347688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.347647    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.347706    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.347762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347780    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.347795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347841    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.34782111 +0000 UTC m=+49.749622238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:49.347862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:49.347892 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347877    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347911    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.347901245 +0000 UTC m=+49.749702354 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347961    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347967    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.347956959 +0000 UTC m=+49.749758077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.347990    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.347981712 +0000 UTC m=+49.749782817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.348041    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.348052    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:49.348217 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:49.348080    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:17:57.348071068 +0000 UTC m=+49.749872178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:52.359643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.359532    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-sdlsb" event={"ID":"114fad46-62e3-4b42-97c0-643871c1b119","Type":"ContainerStarted","Data":"85f8b6f4f4b7576aa90f94427049e4c20fe05b5f09ac9a8e7074e5b4b8eecdc2"}
Sep 10 16:17:52.360129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.359667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:17:52.363108 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.362310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp" event={"ID":"58a190c9-70a4-4c84-96c7-e9c6f4b2214f","Type":"ContainerStarted","Data":"991b68f073045fb6aaf8f4234ff42a2e2d905f0d1a2334d42ea3145d4b5bc9e1"}
Sep 10 16:17:52.367942 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.366718    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q9zm4"]
Sep 10 16:17:52.376673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.376650    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/0.log"
Sep 10 16:17:52.376815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.376691    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7d7b92b-7659-4bf4-a95a-4d5364e3578a" containerID="a6cec7b6e0151ff2f1538b3bfa8098318490db45c2966f4184a13b2aa7a24930" exitCode=255
Sep 10 16:17:52.377000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.376982    2577 scope.go:117] "RemoveContainer" containerID="a6cec7b6e0151ff2f1538b3bfa8098318490db45c2966f4184a13b2aa7a24930"
Sep 10 16:17:52.382249 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xm69q" event={"ID":"9fc228f9-6b63-4f2b-aca1-c4720725d2d0","Type":"ContainerStarted","Data":"97fd4826b3995e302749c04942a4cd85a666c26002f295e53bc14116702ba040"}
Sep 10 16:17:52.382442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9zm4"]
Sep 10 16:17:52.382595 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc" event={"ID":"4f55c1a5-ceb3-4cf2-90d6-55dbd9a1627d","Type":"ContainerStarted","Data":"6020e61cf8424ac3e72e473a6f8a76e230eba7334870a0e8dfe133e9a58e66df"}
Sep 10 16:17:52.382703 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382691    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r" event={"ID":"0d2e7938-d60f-4b3f-9d05-562219253f14","Type":"ContainerStarted","Data":"cf3a64fb6ac964c6fe5592142e44b00bda345a9da54421878d8de1b77e240882"}
Sep 10 16:17:52.382799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" event={"ID":"a7d7b92b-7659-4bf4-a95a-4d5364e3578a","Type":"ContainerDied","Data":"a6cec7b6e0151ff2f1538b3bfa8098318490db45c2966f4184a13b2aa7a24930"}
Sep 10 16:17:52.382904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5" event={"ID":"71fd4ecd-9ead-4a5f-8827-82fbd67d4812","Type":"ContainerStarted","Data":"c4894a0b4f93068ba09a781b73f399ddcdc3809183eda06c5007767b9f5f7d02"}
Sep 10 16:17:52.383006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.382340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.384390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.384336    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-sdlsb" podStartSLOduration=35.953376369 podStartE2EDuration="44.384317801s" totalImagesPullingTime="8.430941432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.381768044 +0000 UTC m=+44.783569173" watchObservedRunningTime="2026-09-10 16:17:52.384317801 +0000 UTC m=+44.786118924"
Sep 10 16:17:52.386113 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.386087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n9wnm\""
Sep 10 16:17:52.386411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.386394    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 16:17:52.386514 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.386469    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 16:17:52.401023 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.400962    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-mqnrc" podStartSLOduration=8.967069257 podStartE2EDuration="17.400946775s" totalImagesPullingTime="8.433877518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.400557649 +0000 UTC m=+44.802358778" watchObservedRunningTime="2026-09-10 16:17:52.400946775 +0000 UTC m=+44.802747903"
Sep 10 16:17:52.421376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.420871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5" podStartSLOduration=8.971833543 podStartE2EDuration="17.420852026s" totalImagesPullingTime="8.449018483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.419405541 +0000 UTC m=+44.821206670" watchObservedRunningTime="2026-09-10 16:17:52.420852026 +0000 UTC m=+44.822653154"
Sep 10 16:17:52.459227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.459172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp" podStartSLOduration=9.007841074 podStartE2EDuration="17.459151956s" totalImagesPullingTime="8.451310882s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.458961163 +0000 UTC m=+44.860762291" watchObservedRunningTime="2026-09-10 16:17:52.459151956 +0000 UTC m=+44.860953084"
Sep 10 16:17:52.475825 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.475794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.475942 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.475908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr29x\" (UniqueName: \"kubernetes.io/projected/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-api-access-wr29x\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.476002 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.475952    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.476077 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.476062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/195d72b1-2a60-4b7b-8591-2a4026d970ec-crio-socket\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.476146 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.476093    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/195d72b1-2a60-4b7b-8591-2a4026d970ec-data-volume\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.503192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.502629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r" podStartSLOduration=9.064968174 podStartE2EDuration="17.502605502s" totalImagesPullingTime="8.437637328s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.479820171 +0000 UTC m=+44.881621299" watchObservedRunningTime="2026-09-10 16:17:52.502605502 +0000 UTC m=+44.904406631"
Sep 10 16:17:52.503423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.503268    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xm69q" podStartSLOduration=3.070099278 podStartE2EDuration="11.503258566s" totalImagesPullingTime="8.433159288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:52.501266493 +0000 UTC m=+44.903067623" watchObservedRunningTime="2026-09-10 16:17:52.503258566 +0000 UTC m=+44.905059694"
Sep 10 16:17:52.576845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.576815    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.576957 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.576911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/195d72b1-2a60-4b7b-8591-2a4026d970ec-crio-socket\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.576957 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.576936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/195d72b1-2a60-4b7b-8591-2a4026d970ec-data-volume\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.577073 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:52.576951    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:17:52.577073 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:52.577031    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:53.077010678 +0000 UTC m=+45.478811790 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : configmap "kube-rbac-proxy" not found
Sep 10 16:17:52.577199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.577073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.577199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.577139    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/195d72b1-2a60-4b7b-8591-2a4026d970ec-crio-socket\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.577199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.577150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr29x\" (UniqueName: \"kubernetes.io/projected/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-api-access-wr29x\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.577373 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:52.577237    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:52.577373 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:52.577314    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:53.077296169 +0000 UTC m=+45.479097289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:52.577373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.577330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/195d72b1-2a60-4b7b-8591-2a4026d970ec-data-volume\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:52.587497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:52.587474    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr29x\" (UniqueName: \"kubernetes.io/projected/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-api-access-wr29x\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:53.082593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.082550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:53.082792 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:53.082669    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:17:53.082792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.082699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:53.082792 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:53.082739    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:54.08271968 +0000 UTC m=+46.484520807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : configmap "kube-rbac-proxy" not found
Sep 10 16:17:53.082963 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:53.082829    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:53.082963 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:53.082892    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:54.082874743 +0000 UTC m=+46.484675867 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:53.366708 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.366671    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"]
Sep 10 16:17:53.384001 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.383969    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:17:53.384531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.384502    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/0.log"
Sep 10 16:17:53.384658 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.384551    2577 generic.go:356] "Generic (PLEG): container finished" podID="a7d7b92b-7659-4bf4-a95a-4d5364e3578a" containerID="8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d" exitCode=255
Sep 10 16:17:53.385567 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.385540    2577 scope.go:117] "RemoveContainer" containerID="8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d"
Sep 10 16:17:53.385739 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:53.385715    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cpn8t_openshift-console-operator(a7d7b92b-7659-4bf4-a95a-4d5364e3578a)\"" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" podUID="a7d7b92b-7659-4bf4-a95a-4d5364e3578a"
Sep 10 16:17:53.401808 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.401782    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"]
Sep 10 16:17:53.401932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.401824    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" event={"ID":"a7d7b92b-7659-4bf4-a95a-4d5364e3578a","Type":"ContainerDied","Data":"8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d"}
Sep 10 16:17:53.401932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.401898    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"
Sep 10 16:17:53.405083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.405063    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 16:17:53.405083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.405075    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 16:17:53.405695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.405678    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-xr7g6\""
Sep 10 16:17:53.411941 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.411914    2577 scope.go:117] "RemoveContainer" containerID="a6cec7b6e0151ff2f1538b3bfa8098318490db45c2966f4184a13b2aa7a24930"
Sep 10 16:17:53.486600 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.486537    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8jpp\" (UniqueName: \"kubernetes.io/projected/e543fad1-4ed6-40f0-bdc7-f084a19fe3bf-kube-api-access-k8jpp\") pod \"migrator-fdbb8fdff-bnx5n\" (UID: \"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"
Sep 10 16:17:53.588897 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.588870    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8jpp\" (UniqueName: \"kubernetes.io/projected/e543fad1-4ed6-40f0-bdc7-f084a19fe3bf-kube-api-access-k8jpp\") pod \"migrator-fdbb8fdff-bnx5n\" (UID: \"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"
Sep 10 16:17:53.597389 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.597362    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8jpp\" (UniqueName: \"kubernetes.io/projected/e543fad1-4ed6-40f0-bdc7-f084a19fe3bf-kube-api-access-k8jpp\") pod \"migrator-fdbb8fdff-bnx5n\" (UID: \"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"
Sep 10 16:17:53.711798 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.711699    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"
Sep 10 16:17:53.730519 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:53.730493    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r84rk_9ab87d95-1588-4a35-9211-d0d53b937dd2/dns-node-resolver/0.log"
Sep 10 16:17:54.010419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.010351    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n"]
Sep 10 16:17:54.013491 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:54.013464    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode543fad1_4ed6_40f0_bdc7_f084a19fe3bf.slice/crio-04c3c66a6e91534e95ccd53ea631590ca7aac6213fb130bf88263638ad797e12 WatchSource:0}: Error finding container 04c3c66a6e91534e95ccd53ea631590ca7aac6213fb130bf88263638ad797e12: Status 404 returned error can't find the container with id 04c3c66a6e91534e95ccd53ea631590ca7aac6213fb130bf88263638ad797e12
Sep 10 16:17:54.095477 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.095440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:54.095637 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.095573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:54.095637 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:54.095571    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:17:54.095709 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:54.095635    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:54.095709 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:54.095669    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:56.095654375 +0000 UTC m=+48.497455480 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : configmap "kube-rbac-proxy" not found
Sep 10 16:17:54.095709 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:54.095690    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:17:56.095676475 +0000 UTC m=+48.497477579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:54.390313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.390265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n" event={"ID":"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf","Type":"ContainerStarted","Data":"04c3c66a6e91534e95ccd53ea631590ca7aac6213fb130bf88263638ad797e12"}
Sep 10 16:17:54.391841 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.391821    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:17:54.392176 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.392159    2577 scope.go:117] "RemoveContainer" containerID="8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d"
Sep 10 16:17:54.392394 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:54.392375    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cpn8t_openshift-console-operator(a7d7b92b-7659-4bf4-a95a-4d5364e3578a)\"" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" podUID="a7d7b92b-7659-4bf4-a95a-4d5364e3578a"
Sep 10 16:17:54.531863 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:54.531837    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d24lj_1500121e-5856-4e8a-96a1-a7815dc5905c/node-ca/0.log"
Sep 10 16:17:55.130667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.130609    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xm69q_9fc228f9-6b63-4f2b-aca1-c4720725d2d0/serve-healthcheck-canary/0.log"
Sep 10 16:17:55.755945 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.755910    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5rk8p"]
Sep 10 16:17:55.768803 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.768773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.769359 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.769336    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5rk8p"]
Sep 10 16:17:55.771866 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.771845    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 16:17:55.772088 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.772065    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 16:17:55.773332 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.773309    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 16:17:55.773459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.773440    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r89r7\""
Sep 10 16:17:55.773559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.773530    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 16:17:55.813101 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.813049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-cabundle\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.813232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.813174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd9r7\" (UniqueName: \"kubernetes.io/projected/a62afd3f-cbbb-469f-a24a-f008ac3ded14-kube-api-access-dd9r7\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.813272 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.813230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-key\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.914527 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.914479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd9r7\" (UniqueName: \"kubernetes.io/projected/a62afd3f-cbbb-469f-a24a-f008ac3ded14-kube-api-access-dd9r7\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.914701 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.914658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-key\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.914759 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.914727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-cabundle\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.915367 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.915347    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-cabundle\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.916927 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.916908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a62afd3f-cbbb-469f-a24a-f008ac3ded14-signing-key\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:55.923677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:55.923657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd9r7\" (UniqueName: \"kubernetes.io/projected/a62afd3f-cbbb-469f-a24a-f008ac3ded14-kube-api-access-dd9r7\") pod \"service-ca-75d99c484-5rk8p\" (UID: \"a62afd3f-cbbb-469f-a24a-f008ac3ded14\") " pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:56.079384 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.079355    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-5rk8p"
Sep 10 16:17:56.115675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.115643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:56.115858 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.115729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:17:56.115858 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:56.115802    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:56.115858 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:56.115849    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 16:17:56.116017 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:56.115875    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:18:00.115855068 +0000 UTC m=+52.517656172 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:17:56.116017 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:56.115901    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:18:00.115885825 +0000 UTC m=+52.517686929 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : configmap "kube-rbac-proxy" not found
Sep 10 16:17:56.372566 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.372540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-5rk8p"]
Sep 10 16:17:56.374690 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:17:56.374663    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda62afd3f_cbbb_469f_a24a_f008ac3ded14.slice/crio-21b9b55b85708dceb3020e55000c8a76d91606f92e86536100cdf15a4248121a WatchSource:0}: Error finding container 21b9b55b85708dceb3020e55000c8a76d91606f92e86536100cdf15a4248121a: Status 404 returned error can't find the container with id 21b9b55b85708dceb3020e55000c8a76d91606f92e86536100cdf15a4248121a
Sep 10 16:17:56.399079 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.399050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-5rk8p" event={"ID":"a62afd3f-cbbb-469f-a24a-f008ac3ded14","Type":"ContainerStarted","Data":"21b9b55b85708dceb3020e55000c8a76d91606f92e86536100cdf15a4248121a"}
Sep 10 16:17:56.400889 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.400866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n" event={"ID":"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf","Type":"ContainerStarted","Data":"2a50b17edf6a321258facb32ca5aa41490d4cb86dd529c82691999877c3a951d"}
Sep 10 16:17:56.400973 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.400907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n" event={"ID":"e543fad1-4ed6-40f0-bdc7-f084a19fe3bf","Type":"ContainerStarted","Data":"1d9c4ae26adbd5aefd9fb76cfc028247638b09a4d708dbf7c13f2384964198cf"}
Sep 10 16:17:56.424518 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:56.424455    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-bnx5n" podStartSLOduration=1.465449922 podStartE2EDuration="3.424433683s" totalImagesPullingTime="1.958983761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:56.423543541 +0000 UTC m=+48.825344669" watchObservedRunningTime="2026-09-10 16:17:56.424433683 +0000 UTC m=+48.826234812"
Sep 10 16:17:57.328089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.328042    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:17:57.328468 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.328234    2577 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:57.328468 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.328347    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls podName:a42b4b80-9ace-4a5f-b0e0-425e2b1c757a nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.328325474 +0000 UTC m=+65.730126582 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-5z5wk" (UID: "a42b4b80-9ace-4a5f-b0e0-425e2b1c757a") : secret "cluster-monitoring-operator-tls" not found
Sep 10 16:17:57.405913 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.405878    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-5rk8p" event={"ID":"a62afd3f-cbbb-469f-a24a-f008ac3ded14","Type":"ContainerStarted","Data":"b13337df6b717568c2d4591006174532b6af7ffca08190257214b7fcdc94414d"}
Sep 10 16:17:57.424074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.424021    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-5rk8p" podStartSLOduration=2.42400582 podStartE2EDuration="2.42400582s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:17:57.423432365 +0000 UTC m=+49.825233495" watchObservedRunningTime="2026-09-10 16:17:57.42400582 +0000 UTC m=+49.825806946"
Sep 10 16:17:57.429415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.429392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:17:57.429530 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.429445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:17:57.429530 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.429489    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:57.429657 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.429538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:17:57.429657 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429597    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 10 16:17:57.429657 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429612    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 10 16:17:57.429657 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429631    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7d574b4464-5m2dj: secret "image-registry-tls" not found
Sep 10 16:17:57.429657 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:17:57.429641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429666    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls podName:b8629aee-eb84-4e99-8b93-259f97a3dbd0 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.429648034 +0000 UTC m=+65.831449153 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-4dhjf" (UID: "b8629aee-eb84-4e99-8b93-259f97a3dbd0") : secret "samples-operator-tls" not found
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429665    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429729    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429735    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls podName:66004ef3-a1df-4c96-bc93-3718aa50d809 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.429704513 +0000 UTC m=+65.831505623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls") pod "image-registry-7d574b4464-5m2dj" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809") : secret "image-registry-tls" not found
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429761    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.429753405 +0000 UTC m=+65.831554510 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : configmap references non-existent config key: service-ca.crt
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429785    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs podName:0094cd2c-52d0-4501-be7d-cb3b6b13b573 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.429771037 +0000 UTC m=+65.831572143 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs") pod "router-default-59648f9df8-wbvrp" (UID: "0094cd2c-52d0-4501-be7d-cb3b6b13b573") : secret "router-metrics-certs-default" not found
Sep 10 16:17:57.429891 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:17:57.429803    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls podName:3f6d7871-16e9-416f-b9ca-6239be4b7fa4 nodeName:}" failed. No retries permitted until 2026-09-10 16:18:13.429794885 +0000 UTC m=+65.831595994 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls") pod "dns-default-tm6h9" (UID: "3f6d7871-16e9-416f-b9ca-6239be4b7fa4") : secret "dns-default-metrics-tls" not found
Sep 10 16:18:00.157842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:00.157806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:00.158326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:00.157860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:00.158326 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:18:00.157953    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 16:18:00.158326 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:18:00.158014    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls podName:195d72b1-2a60-4b7b-8591-2a4026d970ec nodeName:}" failed. No retries permitted until 2026-09-10 16:18:08.157998203 +0000 UTC m=+60.559799307 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q9zm4" (UID: "195d72b1-2a60-4b7b-8591-2a4026d970ec") : secret "insights-runtime-extractor-tls" not found
Sep 10 16:18:00.158507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:00.158380    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/195d72b1-2a60-4b7b-8591-2a4026d970ec-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:01.985487 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:01.985432    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:18:01.985944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:01.985534    2577 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:18:01.985944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:01.985885    2577 scope.go:117] "RemoveContainer" containerID="8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d"
Sep 10 16:18:01.986045 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:18:01.986023    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cpn8t_openshift-console-operator(a7d7b92b-7659-4bf4-a95a-4d5364e3578a)\"" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" podUID="a7d7b92b-7659-4bf4-a95a-4d5364e3578a"
Sep 10 16:18:08.232451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.232416    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:08.234720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.234696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/195d72b1-2a60-4b7b-8591-2a4026d970ec-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q9zm4\" (UID: \"195d72b1-2a60-4b7b-8591-2a4026d970ec\") " pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:08.278916 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.278891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-n746n"
Sep 10 16:18:08.297296 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.297256    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-n9wnm\""
Sep 10 16:18:08.304928 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.304909    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q9zm4"
Sep 10 16:18:08.585608 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:08.585535    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q9zm4"]
Sep 10 16:18:08.588583 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:08.588559    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod195d72b1_2a60_4b7b_8591_2a4026d970ec.slice/crio-270166f7655543f68a186b2b670b0ca4926895a28b84500ad65ccb213935d4c7 WatchSource:0}: Error finding container 270166f7655543f68a186b2b670b0ca4926895a28b84500ad65ccb213935d4c7: Status 404 returned error can't find the container with id 270166f7655543f68a186b2b670b0ca4926895a28b84500ad65ccb213935d4c7
Sep 10 16:18:09.450852 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:09.450821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9zm4" event={"ID":"195d72b1-2a60-4b7b-8591-2a4026d970ec","Type":"ContainerStarted","Data":"2ddf418a10d577e1c6df33e623e27869f9f7ff07180629732bc154c7d3e3b199"}
Sep 10 16:18:09.451227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:09.450907    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9zm4" event={"ID":"195d72b1-2a60-4b7b-8591-2a4026d970ec","Type":"ContainerStarted","Data":"270166f7655543f68a186b2b670b0ca4926895a28b84500ad65ccb213935d4c7"}
Sep 10 16:18:10.455805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:10.455763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9zm4" event={"ID":"195d72b1-2a60-4b7b-8591-2a4026d970ec","Type":"ContainerStarted","Data":"095a318083aec21d61b62dc8b048d75a9b02715ea6b4c2c6ec79085aac2c29df"}
Sep 10 16:18:11.461066 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:11.461031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q9zm4" event={"ID":"195d72b1-2a60-4b7b-8591-2a4026d970ec","Type":"ContainerStarted","Data":"6e8b9ce65d3f4511bd995a00f9fae9a9d54a40116d7d548d1419402553039adb"}
Sep 10 16:18:11.480603 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:11.480560    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q9zm4" podStartSLOduration=17.244136394 podStartE2EDuration="19.480546623s" totalImagesPullingTime="2.236410229s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:11.479792533 +0000 UTC m=+63.881593660" watchObservedRunningTime="2026-09-10 16:18:11.480546623 +0000 UTC m=+63.882347750"
Sep 10 16:18:13.377235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.377191    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:18:13.379748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.379721    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a42b4b80-9ace-4a5f-b0e0-425e2b1c757a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-5z5wk\" (UID: \"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:18:13.478201 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.478162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:18:13.478201 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.478203    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:18:13.478499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.478231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:18:13.478499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.478358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:13.478499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.478409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:13.479147 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.479116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0094cd2c-52d0-4501-be7d-cb3b6b13b573-service-ca-bundle\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:13.480656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.480625    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3f6d7871-16e9-416f-b9ca-6239be4b7fa4-metrics-tls\") pod \"dns-default-tm6h9\" (UID: \"3f6d7871-16e9-416f-b9ca-6239be4b7fa4\") " pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:18:13.480765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.480710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0094cd2c-52d0-4501-be7d-cb3b6b13b573-metrics-certs\") pod \"router-default-59648f9df8-wbvrp\" (UID: \"0094cd2c-52d0-4501-be7d-cb3b6b13b573\") " pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:13.480802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.480762    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/b8629aee-eb84-4e99-8b93-259f97a3dbd0-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-4dhjf\" (UID: \"b8629aee-eb84-4e99-8b93-259f97a3dbd0\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:18:13.480839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.480826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"image-registry-7d574b4464-5m2dj\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") " pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:18:13.594712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.594678    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9j87p\""
Sep 10 16:18:13.602187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.602158    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"
Sep 10 16:18:13.652792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.652759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s2v9s\""
Sep 10 16:18:13.660377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.660348    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:18:13.678710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.678686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-9sw7m\""
Sep 10 16:18:13.686531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.686504    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:13.693827 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.693806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rx6s8\""
Sep 10 16:18:13.700431 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.700384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"
Sep 10 16:18:13.708327 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.708305    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wwcbb\""
Sep 10 16:18:13.715865 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.715817    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:18:13.780832 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.780798    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:18:13.784039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.784017    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 16:18:13.795127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.795072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bdb4c52-c0b9-4029-a2b2-ed39461af0ce-original-pull-secret\") pod \"global-pull-secret-syncer-xwrj7\" (UID: \"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce\") " pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:18:13.881653 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.881566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:18:13.884761 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.884739    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 16:18:13.891070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.891049    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk"]
Sep 10 16:18:13.893177 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:13.893145    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42b4b80_9ace_4a5f_b0e0_425e2b1c757a.slice/crio-a6c45404f854046bfd1198f6de614924e724c8a76db2f4f377ae028b938ff5ea WatchSource:0}: Error finding container a6c45404f854046bfd1198f6de614924e724c8a76db2f4f377ae028b938ff5ea: Status 404 returned error can't find the container with id a6c45404f854046bfd1198f6de614924e724c8a76db2f4f377ae028b938ff5ea
Sep 10 16:18:13.895051 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.895028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13542d90-fc55-4433-aeb2-8ebf3cc57ee0-metrics-certs\") pod \"network-metrics-daemon-87mpg\" (UID: \"13542d90-fc55-4433-aeb2-8ebf3cc57ee0\") " pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:18:13.924729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.924701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xwrj7"
Sep 10 16:18:13.934159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.934125    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j8r7d\""
Sep 10 16:18:13.941715 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.941691    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-87mpg"
Sep 10 16:18:13.957324 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.957295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:18:13.961169 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:13.961096    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66004ef3_a1df_4c96_bc93_3718aa50d809.slice/crio-0a507c34f0ae043844433386d1a2a32c489a051931a1a957752ad184777ac4d0 WatchSource:0}: Error finding container 0a507c34f0ae043844433386d1a2a32c489a051931a1a957752ad184777ac4d0: Status 404 returned error can't find the container with id 0a507c34f0ae043844433386d1a2a32c489a051931a1a957752ad184777ac4d0
Sep 10 16:18:13.994072 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:13.994010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-59648f9df8-wbvrp"]
Sep 10 16:18:13.998086 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:13.998061    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0094cd2c_52d0_4501_be7d_cb3b6b13b573.slice/crio-dce6e22347e12c732f73122dfad4e5838dca05079abfc14e93ae0115018ef194 WatchSource:0}: Error finding container dce6e22347e12c732f73122dfad4e5838dca05079abfc14e93ae0115018ef194: Status 404 returned error can't find the container with id dce6e22347e12c732f73122dfad4e5838dca05079abfc14e93ae0115018ef194
Sep 10 16:18:14.016102 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.014207    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf"]
Sep 10 16:18:14.041884 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.041235    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-tm6h9"]
Sep 10 16:18:14.053326 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:14.053289    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6d7871_16e9_416f_b9ca_6239be4b7fa4.slice/crio-5e03c5d12e8c9360f6f0b165abd79672918ac3fb1b724d62f701b02b5ee4155a WatchSource:0}: Error finding container 5e03c5d12e8c9360f6f0b165abd79672918ac3fb1b724d62f701b02b5ee4155a: Status 404 returned error can't find the container with id 5e03c5d12e8c9360f6f0b165abd79672918ac3fb1b724d62f701b02b5ee4155a
Sep 10 16:18:14.225040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.224944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xwrj7"]
Sep 10 16:18:14.229345 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:14.229312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bdb4c52_c0b9_4029_a2b2_ed39461af0ce.slice/crio-1e54a38b73610b8a49cec1dc8fecdce15afb7f713d2320d89d8557636645f7f1 WatchSource:0}: Error finding container 1e54a38b73610b8a49cec1dc8fecdce15afb7f713d2320d89d8557636645f7f1: Status 404 returned error can't find the container with id 1e54a38b73610b8a49cec1dc8fecdce15afb7f713d2320d89d8557636645f7f1
Sep 10 16:18:14.261612 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.261578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-87mpg"]
Sep 10 16:18:14.266540 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:14.266514    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13542d90_fc55_4433_aeb2_8ebf3cc57ee0.slice/crio-9e7691ce171239ef21d7a5613cb284725b2dc876d57cb1406643f506b8dc6d96 WatchSource:0}: Error finding container 9e7691ce171239ef21d7a5613cb284725b2dc876d57cb1406643f506b8dc6d96: Status 404 returned error can't find the container with id 9e7691ce171239ef21d7a5613cb284725b2dc876d57cb1406643f506b8dc6d96
Sep 10 16:18:14.472597 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.472562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm6h9" event={"ID":"3f6d7871-16e9-416f-b9ca-6239be4b7fa4","Type":"ContainerStarted","Data":"5e03c5d12e8c9360f6f0b165abd79672918ac3fb1b724d62f701b02b5ee4155a"}
Sep 10 16:18:14.473807 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.473785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk" event={"ID":"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a","Type":"ContainerStarted","Data":"a6c45404f854046bfd1198f6de614924e724c8a76db2f4f377ae028b938ff5ea"}
Sep 10 16:18:14.474982 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.474966    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xwrj7" event={"ID":"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce","Type":"ContainerStarted","Data":"1e54a38b73610b8a49cec1dc8fecdce15afb7f713d2320d89d8557636645f7f1"}
Sep 10 16:18:14.476159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.476097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf" event={"ID":"b8629aee-eb84-4e99-8b93-259f97a3dbd0","Type":"ContainerStarted","Data":"a70d0fd8193dfd3e3208c9cbf0d3b0d67309340ccfad9725d5c6de4c34614c80"}
Sep 10 16:18:14.477629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.477602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59648f9df8-wbvrp" event={"ID":"0094cd2c-52d0-4501-be7d-cb3b6b13b573","Type":"ContainerStarted","Data":"c053783ec6b99868546331190ce0d0b28d57f2eb2cc28c539bae217395287914"}
Sep 10 16:18:14.477716 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.477646    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-59648f9df8-wbvrp" event={"ID":"0094cd2c-52d0-4501-be7d-cb3b6b13b573","Type":"ContainerStarted","Data":"dce6e22347e12c732f73122dfad4e5838dca05079abfc14e93ae0115018ef194"}
Sep 10 16:18:14.479121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.479099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" event={"ID":"66004ef3-a1df-4c96-bc93-3718aa50d809","Type":"ContainerStarted","Data":"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"}
Sep 10 16:18:14.479216 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.479138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" event={"ID":"66004ef3-a1df-4c96-bc93-3718aa50d809","Type":"ContainerStarted","Data":"0a507c34f0ae043844433386d1a2a32c489a051931a1a957752ad184777ac4d0"}
Sep 10 16:18:14.479216 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.479192    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:18:14.480320 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.480301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87mpg" event={"ID":"13542d90-fc55-4433-aeb2-8ebf3cc57ee0","Type":"ContainerStarted","Data":"9e7691ce171239ef21d7a5613cb284725b2dc876d57cb1406643f506b8dc6d96"}
Sep 10 16:18:14.500679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.500614    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-59648f9df8-wbvrp" podStartSLOduration=39.500592847 podStartE2EDuration="39.500592847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:14.49884705 +0000 UTC m=+66.900648203" watchObservedRunningTime="2026-09-10 16:18:14.500592847 +0000 UTC m=+66.902393976"
Sep 10 16:18:14.520784 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.520722    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" podStartSLOduration=66.520701513 podStartE2EDuration="1m6.520701513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:14.518726001 +0000 UTC m=+66.920527129" watchObservedRunningTime="2026-09-10 16:18:14.520701513 +0000 UTC m=+66.922502642"
Sep 10 16:18:14.687462 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.687420    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:14.690246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:14.690218    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:15.106840 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.106458    2577 scope.go:117] "RemoveContainer" containerID="8e126edb30286f7d4f9fbb7cc400b704405f35882c9d10ce6b6f87cbce00bd2d"
Sep 10 16:18:15.491302 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.491003    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:18:15.491852 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.491819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" event={"ID":"a7d7b92b-7659-4bf4-a95a-4d5364e3578a","Type":"ContainerStarted","Data":"dec96ce9c95a6f34fb624351155496bf9acb86c9ce99f4c26ef277aa2375a880"}
Sep 10 16:18:15.492819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.492712    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:15.492819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.492757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:18:15.495014 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.494969    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-59648f9df8-wbvrp"
Sep 10 16:18:15.512705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.511527    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-cpn8t" podStartSLOduration=32.060833365 podStartE2EDuration="40.511506668s" totalImagesPullingTime="8.450673303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:15.510832988 +0000 UTC m=+67.912634116" watchObservedRunningTime="2026-09-10 16:18:15.511506668 +0000 UTC m=+67.913307796"
Sep 10 16:18:15.954367 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:15.954328    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-cpn8t"
Sep 10 16:18:18.509411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.508809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87mpg" event={"ID":"13542d90-fc55-4433-aeb2-8ebf3cc57ee0","Type":"ContainerStarted","Data":"452bdd65fdad1b63d2d6f4f1326291277a58fa167329f545b6e1f3695b498e31"}
Sep 10 16:18:18.509411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.508935    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-87mpg" event={"ID":"13542d90-fc55-4433-aeb2-8ebf3cc57ee0","Type":"ContainerStarted","Data":"4f7ad90e79b3f4b033e8239374d169ba5f992374b318a711ddb1b3fc9b6ab367"}
Sep 10 16:18:18.511173 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.511094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm6h9" event={"ID":"3f6d7871-16e9-416f-b9ca-6239be4b7fa4","Type":"ContainerStarted","Data":"9aa2f7c1c1c345514d2d0dbc4ce51f19048c73f61c06846a3052e62beadd8d67"}
Sep 10 16:18:18.511173 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.511149    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-tm6h9" event={"ID":"3f6d7871-16e9-416f-b9ca-6239be4b7fa4","Type":"ContainerStarted","Data":"d9c15e6b9df721db05b8fceed8a97cb96eaa1aef35fb9aeddd037a8443f83fd9"}
Sep 10 16:18:18.511173 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.511179    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:18:18.513012 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.512988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk" event={"ID":"a42b4b80-9ace-4a5f-b0e0-425e2b1c757a","Type":"ContainerStarted","Data":"70b729f8ab267318141caff801afe2958c4fa0a466d99e6bd8b0a2a0276653a2"}
Sep 10 16:18:18.515210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.515181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf" event={"ID":"b8629aee-eb84-4e99-8b93-259f97a3dbd0","Type":"ContainerStarted","Data":"db1c571dc55578b98ccd8d87d3aac24db3cb350c8e8ce1bcf791d08104f8bf70"}
Sep 10 16:18:18.515344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.515231    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf" event={"ID":"b8629aee-eb84-4e99-8b93-259f97a3dbd0","Type":"ContainerStarted","Data":"22d9133dc087dc70cae39dd73e3cd740244b756d64e84dc0b72ad5d0d43bbd2f"}
Sep 10 16:18:18.528893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.528831    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-87mpg" podStartSLOduration=67.32322874 podStartE2EDuration="1m10.528814047s" totalImagesPullingTime="3.205585307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:18.527246318 +0000 UTC m=+70.929047447" watchObservedRunningTime="2026-09-10 16:18:18.528814047 +0000 UTC m=+70.930615174"
Sep 10 16:18:18.570754 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.570672    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-4dhjf" podStartSLOduration=40.188203879 podStartE2EDuration="43.570650983s" totalImagesPullingTime="3.382447104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:18.547585177 +0000 UTC m=+70.949386303" watchObservedRunningTime="2026-09-10 16:18:18.570650983 +0000 UTC m=+70.972452112"
Sep 10 16:18:18.570985 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.570960    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-tm6h9" podStartSLOduration=34.153307685 podStartE2EDuration="37.570949109s" totalImagesPullingTime="3.417641424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:18.569048098 +0000 UTC m=+70.970849274" watchObservedRunningTime="2026-09-10 16:18:18.570949109 +0000 UTC m=+70.972750238"
Sep 10 16:18:18.586021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:18.585629    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-5z5wk" podStartSLOduration=40.003444878 podStartE2EDuration="43.585609207s" totalImagesPullingTime="3.582164329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:17:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:18.585222551 +0000 UTC m=+70.987023682" watchObservedRunningTime="2026-09-10 16:18:18.585609207 +0000 UTC m=+70.987410338"
Sep 10 16:18:19.112756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.111961    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-j95rf"]
Sep 10 16:18:19.116197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.116170    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:19.124515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.124487    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 16:18:19.124688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.124646    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-m8zqq\""
Sep 10 16:18:19.124763 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.124744    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 16:18:19.130215 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.130190    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-j95rf"]
Sep 10 16:18:19.231379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.231290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zr9d8\" (UniqueName: \"kubernetes.io/projected/d73dbe40-0057-41ad-a3fa-c6e668e82709-kube-api-access-zr9d8\") pod \"downloads-85d98c998f-j95rf\" (UID: \"d73dbe40-0057-41ad-a3fa-c6e668e82709\") " pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:19.332460 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.332419    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zr9d8\" (UniqueName: \"kubernetes.io/projected/d73dbe40-0057-41ad-a3fa-c6e668e82709-kube-api-access-zr9d8\") pod \"downloads-85d98c998f-j95rf\" (UID: \"d73dbe40-0057-41ad-a3fa-c6e668e82709\") " pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:19.370237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.370202    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zr9d8\" (UniqueName: \"kubernetes.io/projected/d73dbe40-0057-41ad-a3fa-c6e668e82709-kube-api-access-zr9d8\") pod \"downloads-85d98c998f-j95rf\" (UID: \"d73dbe40-0057-41ad-a3fa-c6e668e82709\") " pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:19.426972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.426939    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:19.728667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:19.728633    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-j95rf"]
Sep 10 16:18:19.732163 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:19.732131    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73dbe40_0057_41ad_a3fa_c6e668e82709.slice/crio-bc1305848bb81d992d3fea156bdeb3e98126efcaf012e17b989fbed2e5e9df8b WatchSource:0}: Error finding container bc1305848bb81d992d3fea156bdeb3e98126efcaf012e17b989fbed2e5e9df8b: Status 404 returned error can't find the container with id bc1305848bb81d992d3fea156bdeb3e98126efcaf012e17b989fbed2e5e9df8b
Sep 10 16:18:20.530688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:20.530650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xwrj7" event={"ID":"0bdb4c52-c0b9-4029-a2b2-ed39461af0ce","Type":"ContainerStarted","Data":"604facb153c361d94a154b25ea314f0607d5e3c2151fc99d54e358059bd392b0"}
Sep 10 16:18:20.531981 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:20.531953    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-j95rf" event={"ID":"d73dbe40-0057-41ad-a3fa-c6e668e82709","Type":"ContainerStarted","Data":"bc1305848bb81d992d3fea156bdeb3e98126efcaf012e17b989fbed2e5e9df8b"}
Sep 10 16:18:20.547431 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:20.547342    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xwrj7" podStartSLOduration=67.313821893 podStartE2EDuration="1m12.547319179s" totalImagesPullingTime="5.233497286s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:20.54655353 +0000 UTC m=+72.948354658" watchObservedRunningTime="2026-09-10 16:18:20.547319179 +0000 UTC m=+72.949120298"
Sep 10 16:18:23.387004 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:23.386968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-sdlsb"
Sep 10 16:18:28.527625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:28.527583    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-tm6h9"
Sep 10 16:18:33.665765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:33.665642    2577 patch_prober.go:28] interesting pod/image-registry-7d574b4464-5m2dj container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 16:18:33.665765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:33.665704    2577 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 16:18:34.553923 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.553817    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s2zpl"]
Sep 10 16:18:34.557545 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.557522    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.561494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.561265    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 16:18:34.561494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.561299    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 16:18:34.561494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.561423    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-gmj9w\""
Sep 10 16:18:34.562398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.562380    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 16:18:34.562907 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.562598    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 16:18:34.660640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660609    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-textfile\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.660822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660649    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-sys\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.660822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.660822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-root\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.660822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.660822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660815    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6v77\" (UniqueName: \"kubernetes.io/projected/f53b5235-c8fe-4048-9f56-3e7d69644030-kube-api-access-n6v77\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.661080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-wtmp\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.661080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-metrics-client-ca\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.661080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.660891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-tls\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762226 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762192    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-textfile\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-sys\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762291    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-sys\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-root\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-root\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762437    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6v77\" (UniqueName: \"kubernetes.io/projected/f53b5235-c8fe-4048-9f56-3e7d69644030-kube-api-access-n6v77\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-wtmp\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762498    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-metrics-client-ca\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.762680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762532    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-tls\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.763171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762717    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-textfile\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.763171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.762799    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-wtmp\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.763422 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.763374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-accelerators-collector-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.764558 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.764530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f53b5235-c8fe-4048-9f56-3e7d69644030-metrics-client-ca\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.765642 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.765618    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-tls\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.771236 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.771191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f53b5235-c8fe-4048-9f56-3e7d69644030-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.772052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.772009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6v77\" (UniqueName: \"kubernetes.io/projected/f53b5235-c8fe-4048-9f56-3e7d69644030-kube-api-access-n6v77\") pod \"node-exporter-s2zpl\" (UID: \"f53b5235-c8fe-4048-9f56-3e7d69644030\") " pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:34.871643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:34.871598    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s2zpl"
Sep 10 16:18:35.498479 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:35.498447    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:18:36.469905 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:36.469876    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf53b5235_c8fe_4048_9f56_3e7d69644030.slice/crio-315eff58113f4ba8676f7f151ee23b5dc6c66d67fc27baea5cb7c5add988c6a6 WatchSource:0}: Error finding container 315eff58113f4ba8676f7f151ee23b5dc6c66d67fc27baea5cb7c5add988c6a6: Status 404 returned error can't find the container with id 315eff58113f4ba8676f7f151ee23b5dc6c66d67fc27baea5cb7c5add988c6a6
Sep 10 16:18:36.607523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.607498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2zpl" event={"ID":"f53b5235-c8fe-4048-9f56-3e7d69644030","Type":"ContainerStarted","Data":"315eff58113f4ba8676f7f151ee23b5dc6c66d67fc27baea5cb7c5add988c6a6"}
Sep 10 16:18:36.609281 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.609257    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-j95rf" event={"ID":"d73dbe40-0057-41ad-a3fa-c6e668e82709","Type":"ContainerStarted","Data":"add178dc22732747735e5e1d3f61e073f54bee614ef8971284901fb5732d1905"}
Sep 10 16:18:36.609511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.609489    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:36.610993 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.610969    2577 patch_prober.go:28] interesting pod/downloads-85d98c998f-j95rf container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused" start-of-body=
Sep 10 16:18:36.611099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.611008    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-85d98c998f-j95rf" podUID="d73dbe40-0057-41ad-a3fa-c6e668e82709" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.21:8080/\": dial tcp 10.132.0.21:8080: connect: connection refused"
Sep 10 16:18:36.630619 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:36.630575    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-j95rf" podStartSLOduration=0.830259545 podStartE2EDuration="17.630560887s" totalImagesPullingTime="16.800301342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:36.62867522 +0000 UTC m=+89.030476346" watchObservedRunningTime="2026-09-10 16:18:36.630560887 +0000 UTC m=+89.032362013"
Sep 10 16:18:37.621478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:37.621444    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-j95rf"
Sep 10 16:18:38.621778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:38.621738    2577 generic.go:356] "Generic (PLEG): container finished" podID="f53b5235-c8fe-4048-9f56-3e7d69644030" containerID="8317b016a8832dab3107e9bf9413e34db6ac5b9aca13a7b7f74e0aff06f65db3" exitCode=0
Sep 10 16:18:38.622237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:38.621839    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2zpl" event={"ID":"f53b5235-c8fe-4048-9f56-3e7d69644030","Type":"ContainerDied","Data":"8317b016a8832dab3107e9bf9413e34db6ac5b9aca13a7b7f74e0aff06f65db3"}
Sep 10 16:18:39.629467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:39.629426    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2zpl" event={"ID":"f53b5235-c8fe-4048-9f56-3e7d69644030","Type":"ContainerStarted","Data":"fc8fe6a6a3e4e407fae2c3acae21c5376b71d8dea5d271d3e5f6f85e77ce396b"}
Sep 10 16:18:39.629909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:39.629492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s2zpl" event={"ID":"f53b5235-c8fe-4048-9f56-3e7d69644030","Type":"ContainerStarted","Data":"49e27962b2e25665f0283547f56f69bcd4304b6027ac8f0bc31e5a51d5d4bea6"}
Sep 10 16:18:39.678206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:39.678135    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s2zpl" podStartSLOduration=4.328860483 podStartE2EDuration="5.678110346s" totalImagesPullingTime="1.241589251s" totalInitContainerRuntime="107.660612ms" isStatefulPod=true podCreationTimestamp="2026-09-10 16:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:39.675569247 +0000 UTC m=+92.077370393" watchObservedRunningTime="2026-09-10 16:18:39.678110346 +0000 UTC m=+92.079911471"
Sep 10 16:18:41.711864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:41.711818    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:18:43.482142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.482103    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:18:43.514443 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.514402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:18:43.514618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.514543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.518791 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.518759    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-qfpmt\""
Sep 10 16:18:43.518946 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.518803    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 16:18:43.519116 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.519091    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 16:18:43.519239 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.519174    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 16:18:43.519239 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.519191    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 16:18:43.519406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.519345    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 16:18:43.524184 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.524155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 16:18:43.547629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.547800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.547800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.547800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547736    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.547800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.548000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jds8v\" (UniqueName: \"kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.548000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.547873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.648975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.648933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.648989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jds8v\" (UniqueName: \"kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649044    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649091    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649125    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.649901 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.650010 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.649875    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.650371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.650349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.650452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.650408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.651952 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.651930    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.653084 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.653044    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.658415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.658387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jds8v\" (UniqueName: \"kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v\") pod \"console-74f44c448d-l5b8s\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") " pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:43.826424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:43.826339    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:44.140294 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:44.140066    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:18:44.143345 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:44.143308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cb85cd6_f20b_4008_bcca_29985181dca9.slice/crio-2d7168e8a5eb93fd731607e6720e6599148fc2e0e461c13f12f0592bbeb9d8f5 WatchSource:0}: Error finding container 2d7168e8a5eb93fd731607e6720e6599148fc2e0e461c13f12f0592bbeb9d8f5: Status 404 returned error can't find the container with id 2d7168e8a5eb93fd731607e6720e6599148fc2e0e461c13f12f0592bbeb9d8f5
Sep 10 16:18:44.662063 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:44.661978    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f44c448d-l5b8s" event={"ID":"0cb85cd6-f20b-4008-bcca-29985181dca9","Type":"ContainerStarted","Data":"2d7168e8a5eb93fd731607e6720e6599148fc2e0e461c13f12f0592bbeb9d8f5"}
Sep 10 16:18:46.356885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.356804    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:18:46.407636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.407603    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:18:46.432341 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.432306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.436878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.436845    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:18:46.476459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7dh2\" (UniqueName: \"kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476594    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476648    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.476773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.476728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577538 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577538 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577659    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7dh2\" (UniqueName: \"kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.577788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.577756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.578713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.578663    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.578713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.578684    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.578869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.578809    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.578869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.578827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.580766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.580746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.581193 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.581167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.587127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.587105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7dh2\" (UniqueName: \"kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2\") pod \"console-769b65fc4d-df4xf\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") " pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:46.745071 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:46.745034    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:47.412766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.412699    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:18:47.447606 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:18:47.447566    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d8a11a_2013_4920_9bea_83df2fc6eada.slice/crio-ed6c9f79add5cdbda9b92ec73397877ebac28cb7deca129b451d1aa4672ddb59 WatchSource:0}: Error finding container ed6c9f79add5cdbda9b92ec73397877ebac28cb7deca129b451d1aa4672ddb59: Status 404 returned error can't find the container with id ed6c9f79add5cdbda9b92ec73397877ebac28cb7deca129b451d1aa4672ddb59
Sep 10 16:18:47.676086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.675987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-769b65fc4d-df4xf" event={"ID":"a2d8a11a-2013-4920-9bea-83df2fc6eada","Type":"ContainerStarted","Data":"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"}
Sep 10 16:18:47.676086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.676056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-769b65fc4d-df4xf" event={"ID":"a2d8a11a-2013-4920-9bea-83df2fc6eada","Type":"ContainerStarted","Data":"ed6c9f79add5cdbda9b92ec73397877ebac28cb7deca129b451d1aa4672ddb59"}
Sep 10 16:18:47.677687 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.677663    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f44c448d-l5b8s" event={"ID":"0cb85cd6-f20b-4008-bcca-29985181dca9","Type":"ContainerStarted","Data":"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"}
Sep 10 16:18:47.699471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.699425    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-769b65fc4d-df4xf" podStartSLOduration=1.699381093 podStartE2EDuration="1.699381093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:18:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:47.69846132 +0000 UTC m=+100.100262447" watchObservedRunningTime="2026-09-10 16:18:47.699381093 +0000 UTC m=+100.101182221"
Sep 10 16:18:47.718841 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:47.718802    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74f44c448d-l5b8s" podStartSLOduration=1.388297514 podStartE2EDuration="4.718790847s" totalImagesPullingTime="3.330493333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:18:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:18:47.718237312 +0000 UTC m=+100.120038431" watchObservedRunningTime="2026-09-10 16:18:47.718790847 +0000 UTC m=+100.120591957"
Sep 10 16:18:53.826757 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:53.826718    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:18:56.745747 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:56.745717    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:56.746116 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:56.745989    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:56.750536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:56.750514    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:18:57.716799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:18:57.716767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:19:02.737613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.737578    2577 generic.go:356] "Generic (PLEG): container finished" podID="58a190c9-70a4-4c84-96c7-e9c6f4b2214f" containerID="991b68f073045fb6aaf8f4234ff42a2e2d905f0d1a2334d42ea3145d4b5bc9e1" exitCode=0
Sep 10 16:19:02.738128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.737655    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp" event={"ID":"58a190c9-70a4-4c84-96c7-e9c6f4b2214f","Type":"ContainerDied","Data":"991b68f073045fb6aaf8f4234ff42a2e2d905f0d1a2334d42ea3145d4b5bc9e1"}
Sep 10 16:19:02.738128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.738054    2577 scope.go:117] "RemoveContainer" containerID="991b68f073045fb6aaf8f4234ff42a2e2d905f0d1a2334d42ea3145d4b5bc9e1"
Sep 10 16:19:02.739219 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.739197    2577 generic.go:356] "Generic (PLEG): container finished" podID="0d2e7938-d60f-4b3f-9d05-562219253f14" containerID="cf3a64fb6ac964c6fe5592142e44b00bda345a9da54421878d8de1b77e240882" exitCode=0
Sep 10 16:19:02.739344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.739243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r" event={"ID":"0d2e7938-d60f-4b3f-9d05-562219253f14","Type":"ContainerDied","Data":"cf3a64fb6ac964c6fe5592142e44b00bda345a9da54421878d8de1b77e240882"}
Sep 10 16:19:02.739504 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:02.739487    2577 scope.go:117] "RemoveContainer" containerID="cf3a64fb6ac964c6fe5592142e44b00bda345a9da54421878d8de1b77e240882"
Sep 10 16:19:03.745833 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:03.745798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-2j8lp" event={"ID":"58a190c9-70a4-4c84-96c7-e9c6f4b2214f","Type":"ContainerStarted","Data":"52e9c65934291a607638e7ff0d5a7392ce1c767a1a22d80ff67aede98be0d1fe"}
Sep 10 16:19:03.750579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:03.750550    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-hln7r" event={"ID":"0d2e7938-d60f-4b3f-9d05-562219253f14","Type":"ContainerStarted","Data":"7591442a86b42c2b1168a372cd1f1c947d95024a64d0ba4d93ad81098f806078"}
Sep 10 16:19:06.735568 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:06.735508    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerName="registry" containerID="cri-o://fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0" gracePeriod=30
Sep 10 16:19:06.971423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:06.971399    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:19:07.061555 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061484    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061555 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061526    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-shj6p\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061555 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061779 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061582    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061779 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061604    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061779 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061646    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061799    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.061970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.061850    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token\") pod \"66004ef3-a1df-4c96-bc93-3718aa50d809\" (UID: \"66004ef3-a1df-4c96-bc93-3718aa50d809\") "
Sep 10 16:19:07.062163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.062135    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:07.062407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.062351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:07.064058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.064023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:07.064354 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.064268    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p" (OuterVolumeSpecName: "kube-api-access-shj6p") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "kube-api-access-shj6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:19:07.064452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.064382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:19:07.064452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.064397    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:07.064569 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.064494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:19:07.070681 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.070655    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "66004ef3-a1df-4c96-bc93-3718aa50d809" (UID: "66004ef3-a1df-4c96-bc93-3718aa50d809"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:19:07.162968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.162928    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-installation-pull-secrets\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.162968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.162960    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-certificates\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.162968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.162973    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-bound-sa-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.163205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.162985    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/66004ef3-a1df-4c96-bc93-3718aa50d809-image-registry-private-configuration\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.163205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.162999    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shj6p\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-kube-api-access-shj6p\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.163205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.163012    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/66004ef3-a1df-4c96-bc93-3718aa50d809-registry-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.163205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.163022    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/66004ef3-a1df-4c96-bc93-3718aa50d809-trusted-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.163205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.163033    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/66004ef3-a1df-4c96-bc93-3718aa50d809-ca-trust-extracted\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:07.766585 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.766552    2577 generic.go:356] "Generic (PLEG): container finished" podID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerID="fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0" exitCode=0
Sep 10 16:19:07.767020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.766587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" event={"ID":"66004ef3-a1df-4c96-bc93-3718aa50d809","Type":"ContainerDied","Data":"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"}
Sep 10 16:19:07.767020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.766610    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj"
Sep 10 16:19:07.767020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.766640    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7d574b4464-5m2dj" event={"ID":"66004ef3-a1df-4c96-bc93-3718aa50d809","Type":"ContainerDied","Data":"0a507c34f0ae043844433386d1a2a32c489a051931a1a957752ad184777ac4d0"}
Sep 10 16:19:07.767020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.766661    2577 scope.go:117] "RemoveContainer" containerID="fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"
Sep 10 16:19:07.775607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.775591    2577 scope.go:117] "RemoveContainer" containerID="fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"
Sep 10 16:19:07.775896 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:19:07.775875    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0\": container with ID starting with fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0 not found: ID does not exist" containerID="fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"
Sep 10 16:19:07.775970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.775915    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0"} err="failed to get container status \"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0\": rpc error: code = NotFound desc = could not find container \"fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0\": container with ID starting with fa67666271944c3374944cdbd44c3270fb936f3c2bfae70b792e1981966171d0 not found: ID does not exist"
Sep 10 16:19:07.789006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.788983    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:19:07.793164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:07.793142    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7d574b4464-5m2dj"]
Sep 10 16:19:08.114305 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:08.114255    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" path="/var/lib/kubelet/pods/66004ef3-a1df-4c96-bc93-3718aa50d809/volumes"
Sep 10 16:19:12.697235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:12.697173    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74f44c448d-l5b8s" podUID="0cb85cd6-f20b-4008-bcca-29985181dca9" containerName="console" containerID="cri-o://fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01" gracePeriod=15
Sep 10 16:19:13.004034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.004001    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:19:13.111123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111086    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111158    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111215    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jds8v\" (UniqueName: \"kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111243    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111412 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111318    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config\") pod \"0cb85cd6-f20b-4008-bcca-29985181dca9\" (UID: \"0cb85cd6-f20b-4008-bcca-29985181dca9\") "
Sep 10 16:19:13.111704 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:13.111847 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:13.111847 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111735    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config" (OuterVolumeSpecName: "console-config") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:13.111951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.111872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca" (OuterVolumeSpecName: "service-ca") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:19:13.113639 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.113608    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v" (OuterVolumeSpecName: "kube-api-access-jds8v") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "kube-api-access-jds8v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:19:13.113750 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.113726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:13.113814 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.113777    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0cb85cd6-f20b-4008-bcca-29985181dca9" (UID: "0cb85cd6-f20b-4008-bcca-29985181dca9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:19:13.212099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212053    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-trusted-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212092    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jds8v\" (UniqueName: \"kubernetes.io/projected/0cb85cd6-f20b-4008-bcca-29985181dca9-kube-api-access-jds8v\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212108    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212121    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-console-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212135    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212148    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0cb85cd6-f20b-4008-bcca-29985181dca9-oauth-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.212398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.212161    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0cb85cd6-f20b-4008-bcca-29985181dca9-console-oauth-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:19:13.790005 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.789966    2577 generic.go:356] "Generic (PLEG): container finished" podID="0cb85cd6-f20b-4008-bcca-29985181dca9" containerID="fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01" exitCode=0
Sep 10 16:19:13.790467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.790031    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f44c448d-l5b8s"
Sep 10 16:19:13.790467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.790068    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f44c448d-l5b8s" event={"ID":"0cb85cd6-f20b-4008-bcca-29985181dca9","Type":"ContainerDied","Data":"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"}
Sep 10 16:19:13.790467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.790141    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f44c448d-l5b8s" event={"ID":"0cb85cd6-f20b-4008-bcca-29985181dca9","Type":"ContainerDied","Data":"2d7168e8a5eb93fd731607e6720e6599148fc2e0e461c13f12f0592bbeb9d8f5"}
Sep 10 16:19:13.790467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.790162    2577 scope.go:117] "RemoveContainer" containerID="fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"
Sep 10 16:19:13.798362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.798344    2577 scope.go:117] "RemoveContainer" containerID="fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"
Sep 10 16:19:13.798650 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:19:13.798632    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01\": container with ID starting with fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01 not found: ID does not exist" containerID="fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"
Sep 10 16:19:13.798688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.798659    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01"} err="failed to get container status \"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01\": rpc error: code = NotFound desc = could not find container \"fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01\": container with ID starting with fcdb101567cac98e4c69ed7c690e32432e7a9cad7a37507ca878ab8f4b772c01 not found: ID does not exist"
Sep 10 16:19:13.811459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.811436    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:19:13.815164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:13.815145    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74f44c448d-l5b8s"]
Sep 10 16:19:14.111293 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:14.111198    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cb85cd6-f20b-4008-bcca-29985181dca9" path="/var/lib/kubelet/pods/0cb85cd6-f20b-4008-bcca-29985181dca9/volumes"
Sep 10 16:19:27.844315 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:27.844257    2577 generic.go:356] "Generic (PLEG): container finished" podID="71fd4ecd-9ead-4a5f-8827-82fbd67d4812" containerID="c4894a0b4f93068ba09a781b73f399ddcdc3809183eda06c5007767b9f5f7d02" exitCode=0
Sep 10 16:19:27.844718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:27.844334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5" event={"ID":"71fd4ecd-9ead-4a5f-8827-82fbd67d4812","Type":"ContainerDied","Data":"c4894a0b4f93068ba09a781b73f399ddcdc3809183eda06c5007767b9f5f7d02"}
Sep 10 16:19:27.844767 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:27.844739    2577 scope.go:117] "RemoveContainer" containerID="c4894a0b4f93068ba09a781b73f399ddcdc3809183eda06c5007767b9f5f7d02"
Sep 10 16:19:28.849890 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:28.849852    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-ctsf5" event={"ID":"71fd4ecd-9ead-4a5f-8827-82fbd67d4812","Type":"ContainerStarted","Data":"0a6c23bd12d59ed429d866b09915cf66413dc0168b5820fa5f64102c33af3150"}
Sep 10 16:19:55.722573 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722535    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722877    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerName="registry"
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722891    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerName="registry"
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722926    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cb85cd6-f20b-4008-bcca-29985181dca9" containerName="console"
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722932    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cb85cd6-f20b-4008-bcca-29985181dca9" containerName="console"
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722976    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="66004ef3-a1df-4c96-bc93-3718aa50d809" containerName="registry"
Sep 10 16:19:55.723045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.722984    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cb85cd6-f20b-4008-bcca-29985181dca9" containerName="console"
Sep 10 16:19:55.725867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.725847    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.740612 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.740589    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:19:55.838969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.838935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839003    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zthm\" (UniqueName: \"kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839058    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.839342 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.839166    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940239    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zthm\" (UniqueName: \"kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940323    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.940650 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.940456    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.941119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.941089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.941231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.941099    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.941231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.941147    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.941334 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.941225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.942905 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.942885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.942991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.942929    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:55.949118 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:55.949098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zthm\" (UniqueName: \"kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm\") pod \"console-d878897b-bzbl5\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") " pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:56.037433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:56.037350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:19:56.310845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:56.310820    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:19:56.313387 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:19:56.313360    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a996ff0_2ba4_49b4_af18_146e6dd6980f.slice/crio-935b6f130abab407f090dcf76f4441870e890060e00b547e309da1d00ce661d8 WatchSource:0}: Error finding container 935b6f130abab407f090dcf76f4441870e890060e00b547e309da1d00ce661d8: Status 404 returned error can't find the container with id 935b6f130abab407f090dcf76f4441870e890060e00b547e309da1d00ce661d8
Sep 10 16:19:56.958333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:56.958301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d878897b-bzbl5" event={"ID":"6a996ff0-2ba4-49b4-af18-146e6dd6980f","Type":"ContainerStarted","Data":"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"}
Sep 10 16:19:56.958693 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:56.958350    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d878897b-bzbl5" event={"ID":"6a996ff0-2ba4-49b4-af18-146e6dd6980f","Type":"ContainerStarted","Data":"935b6f130abab407f090dcf76f4441870e890060e00b547e309da1d00ce661d8"}
Sep 10 16:19:56.979795 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:19:56.979747    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d878897b-bzbl5" podStartSLOduration=1.9797314940000001 podStartE2EDuration="1.979731494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:19:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:19:56.979322086 +0000 UTC m=+169.381123212" watchObservedRunningTime="2026-09-10 16:19:56.979731494 +0000 UTC m=+169.381532621"
Sep 10 16:20:06.038125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:06.038011    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:20:06.038125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:06.038131    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:20:06.042766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:06.042746    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:20:06.998802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:06.998773    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:20:07.079420 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:07.079391    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:20:32.099565 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.099502    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-769b65fc4d-df4xf" podUID="a2d8a11a-2013-4920-9bea-83df2fc6eada" containerName="console" containerID="cri-o://b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d" gracePeriod=15
Sep 10 16:20:32.337387 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.337363    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:20:32.349471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349487    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349516    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349676    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7dh2\" (UniqueName: \"kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349745    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349843    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config\") pod \"a2d8a11a-2013-4920-9bea-83df2fc6eada\" (UID: \"a2d8a11a-2013-4920-9bea-83df2fc6eada\") "
Sep 10 16:20:32.349925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349844    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca" (OuterVolumeSpecName: "service-ca") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:32.349925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:32.349925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.349913    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:32.350106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.350039    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-trusted-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.350106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.350063    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-oauth-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.350106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.350081    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.350194 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.350148    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config" (OuterVolumeSpecName: "console-config") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:20:32.351703 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.351675    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:32.351809 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.351678    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2" (OuterVolumeSpecName: "kube-api-access-f7dh2") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "kube-api-access-f7dh2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:20:32.351850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.351834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a2d8a11a-2013-4920-9bea-83df2fc6eada" (UID: "a2d8a11a-2013-4920-9bea-83df2fc6eada"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:20:32.451268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.451237    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.451268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.451261    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-oauth-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.451268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.451288    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2d8a11a-2013-4920-9bea-83df2fc6eada-console-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:32.451503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:32.451299    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7dh2\" (UniqueName: \"kubernetes.io/projected/a2d8a11a-2013-4920-9bea-83df2fc6eada-kube-api-access-f7dh2\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:20:33.094720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.094686    2577 generic.go:356] "Generic (PLEG): container finished" podID="a2d8a11a-2013-4920-9bea-83df2fc6eada" containerID="b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d" exitCode=0
Sep 10 16:20:33.094900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.094733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-769b65fc4d-df4xf"
Sep 10 16:20:33.094900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.094788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-769b65fc4d-df4xf" event={"ID":"a2d8a11a-2013-4920-9bea-83df2fc6eada","Type":"ContainerDied","Data":"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"}
Sep 10 16:20:33.094900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.094833    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-769b65fc4d-df4xf" event={"ID":"a2d8a11a-2013-4920-9bea-83df2fc6eada","Type":"ContainerDied","Data":"ed6c9f79add5cdbda9b92ec73397877ebac28cb7deca129b451d1aa4672ddb59"}
Sep 10 16:20:33.094900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.094855    2577 scope.go:117] "RemoveContainer" containerID="b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"
Sep 10 16:20:33.104045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.103850    2577 scope.go:117] "RemoveContainer" containerID="b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"
Sep 10 16:20:33.104310 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:20:33.104156    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d\": container with ID starting with b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d not found: ID does not exist" containerID="b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"
Sep 10 16:20:33.104310 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.104194    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d"} err="failed to get container status \"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d\": rpc error: code = NotFound desc = could not find container \"b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d\": container with ID starting with b35e9603efba264b42b269458819c9c0ac3981a0644685743dd9bd18d57f674d not found: ID does not exist"
Sep 10 16:20:33.121723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.121695    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:20:33.126359 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:33.126336    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-769b65fc4d-df4xf"]
Sep 10 16:20:34.110769 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:20:34.110734    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d8a11a-2013-4920-9bea-83df2fc6eada" path="/var/lib/kubelet/pods/a2d8a11a-2013-4920-9bea-83df2fc6eada/volumes"
Sep 10 16:21:11.456673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.456641    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"]
Sep 10 16:21:11.457067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.456939    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d8a11a-2013-4920-9bea-83df2fc6eada" containerName="console"
Sep 10 16:21:11.457067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.456951    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d8a11a-2013-4920-9bea-83df2fc6eada" containerName="console"
Sep 10 16:21:11.457067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.457013    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d8a11a-2013-4920-9bea-83df2fc6eada" containerName="console"
Sep 10 16:21:11.459350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.459333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.462268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.462245    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 16:21:11.462588 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.462572    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skc8x\""
Sep 10 16:21:11.463675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.463659    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 16:21:11.476731 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.476704    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"]
Sep 10 16:21:11.557164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.557127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.557377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.557186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrnmc\" (UniqueName: \"kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.557377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.557305    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.657634 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.657602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.657821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.657649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrnmc\" (UniqueName: \"kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.657821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.657773    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.658034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.658011    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.658096 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.658040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.667013 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.666989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrnmc\" (UniqueName: \"kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:11.772951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:11.772859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:12.044442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:12.044379    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"]
Sep 10 16:21:12.046494 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:21:12.046466    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod076f98c5_c6ce_41d4_b2d7_1471ead96801.slice/crio-11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4 WatchSource:0}: Error finding container 11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4: Status 404 returned error can't find the container with id 11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4
Sep 10 16:21:12.236212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:12.236178    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj" event={"ID":"076f98c5-c6ce-41d4-b2d7-1471ead96801","Type":"ContainerStarted","Data":"11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4"}
Sep 10 16:21:19.265757 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:19.265671    2577 generic.go:356] "Generic (PLEG): container finished" podID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerID="a9cb4561321b1ebbfc7de754b3b2567f2e718b21fa21a79fc1cb43c9d6cc4e6c" exitCode=0
Sep 10 16:21:19.266137 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:19.265759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj" event={"ID":"076f98c5-c6ce-41d4-b2d7-1471ead96801","Type":"ContainerDied","Data":"a9cb4561321b1ebbfc7de754b3b2567f2e718b21fa21a79fc1cb43c9d6cc4e6c"}
Sep 10 16:21:22.279954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:22.279918    2577 generic.go:356] "Generic (PLEG): container finished" podID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerID="9dca213d1f95c0e98b9f2a12dfbe6cd7caa1282fccba94a9142e8e186bc8ea81" exitCode=0
Sep 10 16:21:22.279954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:22.279955    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj" event={"ID":"076f98c5-c6ce-41d4-b2d7-1471ead96801","Type":"ContainerDied","Data":"9dca213d1f95c0e98b9f2a12dfbe6cd7caa1282fccba94a9142e8e186bc8ea81"}
Sep 10 16:21:29.313355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:29.313315    2577 generic.go:356] "Generic (PLEG): container finished" podID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerID="7fe7cd37deaab0e878e21fce875931a149f5af786b14285ed0457c3b95a0434f" exitCode=0
Sep 10 16:21:29.313792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:29.313379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj" event={"ID":"076f98c5-c6ce-41d4-b2d7-1471ead96801","Type":"ContainerDied","Data":"7fe7cd37deaab0e878e21fce875931a149f5af786b14285ed0457c3b95a0434f"}
Sep 10 16:21:30.436437 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.436415    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:30.617618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.617590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrnmc\" (UniqueName: \"kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc\") pod \"076f98c5-c6ce-41d4-b2d7-1471ead96801\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") "
Sep 10 16:21:30.617777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.617633    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util\") pod \"076f98c5-c6ce-41d4-b2d7-1471ead96801\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") "
Sep 10 16:21:30.617777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.617674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle\") pod \"076f98c5-c6ce-41d4-b2d7-1471ead96801\" (UID: \"076f98c5-c6ce-41d4-b2d7-1471ead96801\") "
Sep 10 16:21:30.618427 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.618400    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle" (OuterVolumeSpecName: "bundle") pod "076f98c5-c6ce-41d4-b2d7-1471ead96801" (UID: "076f98c5-c6ce-41d4-b2d7-1471ead96801"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:30.619776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.619748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc" (OuterVolumeSpecName: "kube-api-access-rrnmc") pod "076f98c5-c6ce-41d4-b2d7-1471ead96801" (UID: "076f98c5-c6ce-41d4-b2d7-1471ead96801"). InnerVolumeSpecName "kube-api-access-rrnmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:21:30.622198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.622171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util" (OuterVolumeSpecName: "util") pod "076f98c5-c6ce-41d4-b2d7-1471ead96801" (UID: "076f98c5-c6ce-41d4-b2d7-1471ead96801"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:30.718383 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.718349    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:30.718383 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.718376    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrnmc\" (UniqueName: \"kubernetes.io/projected/076f98c5-c6ce-41d4-b2d7-1471ead96801-kube-api-access-rrnmc\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:30.718383 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:30.718386    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/076f98c5-c6ce-41d4-b2d7-1471ead96801-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:31.322746 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:31.322716    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj"
Sep 10 16:21:31.322911 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:31.322715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407knhj" event={"ID":"076f98c5-c6ce-41d4-b2d7-1471ead96801","Type":"ContainerDied","Data":"11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4"}
Sep 10 16:21:31.322911 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:31.322837    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11aac21801364aff8217213d8555d6d2a6a480e56fdbe1d740a0f0c69a7a7af4"
Sep 10 16:21:34.976586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976553    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"]
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976826    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="extract"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976837    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="extract"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976845    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="util"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976851    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="util"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976865    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="pull"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976871    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="pull"
Sep 10 16:21:34.976969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.976912    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="076f98c5-c6ce-41d4-b2d7-1471ead96801" containerName="extract"
Sep 10 16:21:34.998086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.998054    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:34.998562 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:34.998463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"]
Sep 10 16:21:35.001039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.001018    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 16:21:35.002581 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.002563    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-67z47\""
Sep 10 16:21:35.002753 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.002736    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 16:21:35.048548 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.048516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbmkp\" (UniqueName: \"kubernetes.io/projected/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-kube-api-access-mbmkp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.048711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.048556    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.149256 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.149217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbmkp\" (UniqueName: \"kubernetes.io/projected/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-kube-api-access-mbmkp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.149437 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.149271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.149604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.149590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.159882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.159840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbmkp\" (UniqueName: \"kubernetes.io/projected/ec75b4cf-1d5d-4727-b1de-eb8d7961084f-kube-api-access-mbmkp\") pod \"cert-manager-operator-controller-manager-55c444bf86-mxmz4\" (UID: \"ec75b4cf-1d5d-4727-b1de-eb8d7961084f\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.310344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.310246    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"
Sep 10 16:21:35.611091 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:35.610978    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4"]
Sep 10 16:21:35.613427 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:21:35.613393    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec75b4cf_1d5d_4727_b1de_eb8d7961084f.slice/crio-cbdf7d0746072fab38fc4ca22a412a4de72614615d7eb87856fa19e774e0c12f WatchSource:0}: Error finding container cbdf7d0746072fab38fc4ca22a412a4de72614615d7eb87856fa19e774e0c12f: Status 404 returned error can't find the container with id cbdf7d0746072fab38fc4ca22a412a4de72614615d7eb87856fa19e774e0c12f
Sep 10 16:21:36.343043 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:36.343006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4" event={"ID":"ec75b4cf-1d5d-4727-b1de-eb8d7961084f","Type":"ContainerStarted","Data":"cbdf7d0746072fab38fc4ca22a412a4de72614615d7eb87856fa19e774e0c12f"}
Sep 10 16:21:38.353875 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:38.353776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4" event={"ID":"ec75b4cf-1d5d-4727-b1de-eb8d7961084f","Type":"ContainerStarted","Data":"2dca8e906597198adc60372969f557ee09ed6ab758515a071dc4f95439bce171"}
Sep 10 16:21:38.380182 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:38.380123    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-mxmz4" podStartSLOduration=2.058977401 podStartE2EDuration="4.3801092s" totalImagesPullingTime="2.321131799s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:38.378852224 +0000 UTC m=+270.780653348" watchObservedRunningTime="2026-09-10 16:21:38.3801092 +0000 UTC m=+270.781910327"
Sep 10 16:21:39.202032 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.201998    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"]
Sep 10 16:21:39.223600 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.223570    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.224978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.224951    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"]
Sep 10 16:21:39.227175 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.227152    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 16:21:39.227306 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.227180    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 16:21:39.228540 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.228517    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skc8x\""
Sep 10 16:21:39.282242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.282212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.282421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.282249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.282421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.282378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5l7rs\" (UniqueName: \"kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.334723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.334690    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wkwrv"]
Sep 10 16:21:39.357478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.357449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wkwrv"]
Sep 10 16:21:39.357888 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.357566    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.361815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.361788    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-5c45s\""
Sep 10 16:21:39.361961 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.361949    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 10 16:21:39.362031 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.362009    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 10 16:21:39.383497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383472    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5l7rs\" (UniqueName: \"kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.383622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2lzr4\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-kube-api-access-2lzr4\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.383622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.383748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383727    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.383894 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.383993 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.383974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.384033 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.384016    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.403619 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.403592    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5l7rs\" (UniqueName: \"kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.485098 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.485010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2lzr4\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-kube-api-access-2lzr4\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.485098 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.485078    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.504445 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.504413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2lzr4\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-kube-api-access-2lzr4\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.505564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.505545    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-wkwrv\" (UID: \"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6\") " pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.534497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.534469    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:39.680576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.680541    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:39.829243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.829216    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"]
Sep 10 16:21:39.831286 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:21:39.831250    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc336e04b_682c_4420_a2ba_250c728cbdc1.slice/crio-b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19 WatchSource:0}: Error finding container b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19: Status 404 returned error can't find the container with id b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19
Sep 10 16:21:39.977899 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:39.977833    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-wkwrv"]
Sep 10 16:21:39.980214 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:21:39.980190    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3847067d_c3e3_4cec_bf9e_2ef26bb0d5d6.slice/crio-a092f5e7c88f03ae64618965fd42ea5422bb7d6cef23a1b1733effc6f9a53a9a WatchSource:0}: Error finding container a092f5e7c88f03ae64618965fd42ea5422bb7d6cef23a1b1733effc6f9a53a9a: Status 404 returned error can't find the container with id a092f5e7c88f03ae64618965fd42ea5422bb7d6cef23a1b1733effc6f9a53a9a
Sep 10 16:21:40.364419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:40.364386    2577 generic.go:356] "Generic (PLEG): container finished" podID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerID="36134b9ad5297fba7ddea4726c0f92f767106ac3783183135ee57ed1cbb2746e" exitCode=0
Sep 10 16:21:40.364915 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:40.364477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j" event={"ID":"c336e04b-682c-4420-a2ba-250c728cbdc1","Type":"ContainerDied","Data":"36134b9ad5297fba7ddea4726c0f92f767106ac3783183135ee57ed1cbb2746e"}
Sep 10 16:21:40.364915 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:40.364526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j" event={"ID":"c336e04b-682c-4420-a2ba-250c728cbdc1","Type":"ContainerStarted","Data":"b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19"}
Sep 10 16:21:40.365820 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:40.365787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv" event={"ID":"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6","Type":"ContainerStarted","Data":"a092f5e7c88f03ae64618965fd42ea5422bb7d6cef23a1b1733effc6f9a53a9a"}
Sep 10 16:21:44.386922 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:44.386884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv" event={"ID":"3847067d-c3e3-4cec-bf9e-2ef26bb0d5d6","Type":"ContainerStarted","Data":"11d0b98420394c26124894f07ddf3b7637a945b91b426f79ca8960a9387d6ade"}
Sep 10 16:21:44.387430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:44.386963    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:44.388709 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:44.388682    2577 generic.go:356] "Generic (PLEG): container finished" podID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerID="d2fa85ef3828f693c608b777b13904ac62d785e06884dca54b65c4175ebd1e2a" exitCode=0
Sep 10 16:21:44.388814 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:44.388746    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j" event={"ID":"c336e04b-682c-4420-a2ba-250c728cbdc1","Type":"ContainerDied","Data":"d2fa85ef3828f693c608b777b13904ac62d785e06884dca54b65c4175ebd1e2a"}
Sep 10 16:21:44.407371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:44.407323    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv" podStartSLOduration=1.826901973 podStartE2EDuration="5.407306057s" totalImagesPullingTime="3.580404084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:44.405821613 +0000 UTC m=+276.807622752" watchObservedRunningTime="2026-09-10 16:21:44.407306057 +0000 UTC m=+276.809107181"
Sep 10 16:21:45.395235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:45.395203    2577 generic.go:356] "Generic (PLEG): container finished" podID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerID="799278df5f27ae5b60b2da5eef14fda2efd523af85f07f4fba392fecf979fa83" exitCode=0
Sep 10 16:21:45.395623 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:45.395301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j" event={"ID":"c336e04b-682c-4420-a2ba-250c728cbdc1","Type":"ContainerDied","Data":"799278df5f27ae5b60b2da5eef14fda2efd523af85f07f4fba392fecf979fa83"}
Sep 10 16:21:46.528931 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.528905    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:46.643503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.643474    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle\") pod \"c336e04b-682c-4420-a2ba-250c728cbdc1\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") "
Sep 10 16:21:46.643503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.643517    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util\") pod \"c336e04b-682c-4420-a2ba-250c728cbdc1\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") "
Sep 10 16:21:46.643756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.643544    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5l7rs\" (UniqueName: \"kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs\") pod \"c336e04b-682c-4420-a2ba-250c728cbdc1\" (UID: \"c336e04b-682c-4420-a2ba-250c728cbdc1\") "
Sep 10 16:21:46.643945 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.643917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle" (OuterVolumeSpecName: "bundle") pod "c336e04b-682c-4420-a2ba-250c728cbdc1" (UID: "c336e04b-682c-4420-a2ba-250c728cbdc1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:46.645686 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.645656    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs" (OuterVolumeSpecName: "kube-api-access-5l7rs") pod "c336e04b-682c-4420-a2ba-250c728cbdc1" (UID: "c336e04b-682c-4420-a2ba-250c728cbdc1"). InnerVolumeSpecName "kube-api-access-5l7rs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:21:46.648917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.648885    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util" (OuterVolumeSpecName: "util") pod "c336e04b-682c-4420-a2ba-250c728cbdc1" (UID: "c336e04b-682c-4420-a2ba-250c728cbdc1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:21:46.745026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.744941    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:46.745026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.744969    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c336e04b-682c-4420-a2ba-250c728cbdc1-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:46.745026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:46.744980    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5l7rs\" (UniqueName: \"kubernetes.io/projected/c336e04b-682c-4420-a2ba-250c728cbdc1-kube-api-access-5l7rs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:21:47.405751 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:47.405721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j"
Sep 10 16:21:47.405751 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:47.405731    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f6z74j" event={"ID":"c336e04b-682c-4420-a2ba-250c728cbdc1","Type":"ContainerDied","Data":"b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19"}
Sep 10 16:21:47.405967 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:47.405774    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b39a2489896e256924fae37f02ad0bf9775ed484edb31c839afbe5efe6f25e19"
Sep 10 16:21:50.397695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.397663    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-wkwrv"
Sep 10 16:21:50.468119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcvp7"]
Sep 10 16:21:50.468506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468487    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="util"
Sep 10 16:21:50.468564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468509    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="util"
Sep 10 16:21:50.468564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468535    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="extract"
Sep 10 16:21:50.468564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468544    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="extract"
Sep 10 16:21:50.468564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468556    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="pull"
Sep 10 16:21:50.468564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468564    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="pull"
Sep 10 16:21:50.468725 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.468646    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c336e04b-682c-4420-a2ba-250c728cbdc1" containerName="extract"
Sep 10 16:21:50.470796 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.470777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.473695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.473673    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-cbmrb\""
Sep 10 16:21:50.482416 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.482394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcvp7"]
Sep 10 16:21:50.574082 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.574040    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.574082 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.574084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rvmd\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-kube-api-access-8rvmd\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.675503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.675393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.675503 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.675449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rvmd\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-kube-api-access-8rvmd\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.684449 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.684414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-bound-sa-token\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.684639 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.684621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rvmd\" (UniqueName: \"kubernetes.io/projected/445fbef2-ba10-4de4-bff8-5920bff03da6-kube-api-access-8rvmd\") pod \"cert-manager-55b4d954d5-mcvp7\" (UID: \"445fbef2-ba10-4de4-bff8-5920bff03da6\") " pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:50.779764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:50.779727    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-mcvp7"
Sep 10 16:21:51.055295 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:51.055191    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-mcvp7"]
Sep 10 16:21:51.058450 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:21:51.058415    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod445fbef2_ba10_4de4_bff8_5920bff03da6.slice/crio-f72e99e3b53ef156cf209e5098c61cef9969acdce02a02af84cf8931872b6558 WatchSource:0}: Error finding container f72e99e3b53ef156cf209e5098c61cef9969acdce02a02af84cf8931872b6558: Status 404 returned error can't find the container with id f72e99e3b53ef156cf209e5098c61cef9969acdce02a02af84cf8931872b6558
Sep 10 16:21:51.423868 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:51.423829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mcvp7" event={"ID":"445fbef2-ba10-4de4-bff8-5920bff03da6","Type":"ContainerStarted","Data":"426288e85b4290b4de7da4cb81e52d83be1460a9f731cbdb93e729e04812a12e"}
Sep 10 16:21:51.424235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:51.423876    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-mcvp7" event={"ID":"445fbef2-ba10-4de4-bff8-5920bff03da6","Type":"ContainerStarted","Data":"f72e99e3b53ef156cf209e5098c61cef9969acdce02a02af84cf8931872b6558"}
Sep 10 16:21:51.450474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:21:51.450420    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-mcvp7" podStartSLOduration=1.450404936 podStartE2EDuration="1.450404936s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:21:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:21:51.449036748 +0000 UTC m=+283.850837874" watchObservedRunningTime="2026-09-10 16:21:51.450404936 +0000 UTC m=+283.852206063"
Sep 10 16:22:01.644951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.644910    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"]
Sep 10 16:22:01.648451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.648429    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.656951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.656931    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skc8x\""
Sep 10 16:22:01.657245 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.657230    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 16:22:01.658271 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.658257    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 16:22:01.667627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.667600    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"]
Sep 10 16:22:01.758136 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.758094    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.758136 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.758143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.758381 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.758225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htxcm\" (UniqueName: \"kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.859567 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.859527    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.859567 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.859574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.859821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.859593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htxcm\" (UniqueName: \"kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.859978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.859954    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.860042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.859995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.869233 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.869195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htxcm\" (UniqueName: \"kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:01.957422 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:01.957333    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:02.249522 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:02.249448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"]
Sep 10 16:22:02.252529 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:02.252504    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37a59b9d_eb29_4a1f_8adc_59a4d8093ffd.slice/crio-f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2 WatchSource:0}: Error finding container f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2: Status 404 returned error can't find the container with id f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2
Sep 10 16:22:02.472080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:02.472043    2577 generic.go:356] "Generic (PLEG): container finished" podID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerID="ff2949cc8066cb3818580784d18d3a70c42b2a9ebcb19cf9f9b70f0ec7d0727d" exitCode=0
Sep 10 16:22:02.472243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:02.472132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm" event={"ID":"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd","Type":"ContainerDied","Data":"ff2949cc8066cb3818580784d18d3a70c42b2a9ebcb19cf9f9b70f0ec7d0727d"}
Sep 10 16:22:02.472243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:02.472189    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm" event={"ID":"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd","Type":"ContainerStarted","Data":"f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2"}
Sep 10 16:22:03.478867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:03.478789    2577 generic.go:356] "Generic (PLEG): container finished" podID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerID="403eeebd2730a8e95c0d08f5f5a81fc247742717659ec443a1236c07f71a6276" exitCode=0
Sep 10 16:22:03.478867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:03.478825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm" event={"ID":"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd","Type":"ContainerDied","Data":"403eeebd2730a8e95c0d08f5f5a81fc247742717659ec443a1236c07f71a6276"}
Sep 10 16:22:04.485855 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:04.485819    2577 generic.go:356] "Generic (PLEG): container finished" podID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerID="95a8d409d6d9bbec28988fcb212232b36192267f8879dac7efa736c0451d8cc3" exitCode=0
Sep 10 16:22:04.486230 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:04.485871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm" event={"ID":"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd","Type":"ContainerDied","Data":"95a8d409d6d9bbec28988fcb212232b36192267f8879dac7efa736c0451d8cc3"}
Sep 10 16:22:05.618049 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.618024    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:05.690426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.690390    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle\") pod \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") "
Sep 10 16:22:05.690426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.690428    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util\") pod \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") "
Sep 10 16:22:05.690638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.690506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htxcm\" (UniqueName: \"kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm\") pod \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\" (UID: \"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd\") "
Sep 10 16:22:05.691146 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.691123    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle" (OuterVolumeSpecName: "bundle") pod "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" (UID: "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:05.692668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.692641    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm" (OuterVolumeSpecName: "kube-api-access-htxcm") pod "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" (UID: "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd"). InnerVolumeSpecName "kube-api-access-htxcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:05.696189 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.696164    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util" (OuterVolumeSpecName: "util") pod "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" (UID: "37a59b9d-eb29-4a1f-8adc-59a4d8093ffd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:05.791859 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.791785    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htxcm\" (UniqueName: \"kubernetes.io/projected/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-kube-api-access-htxcm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:05.791859 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.791811    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:05.791859 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:05.791821    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/37a59b9d-eb29-4a1f-8adc-59a4d8093ffd-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:06.496851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:06.496768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm" event={"ID":"37a59b9d-eb29-4a1f-8adc-59a4d8093ffd","Type":"ContainerDied","Data":"f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2"}
Sep 10 16:22:06.496851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:06.496846    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f293e84888d33d0b76b2f09dd56e9080c3ffb13b2ae5c6966a536a7a32b05cd2"
Sep 10 16:22:06.497026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:06.496777    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxdzqm"
Sep 10 16:22:08.018934 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:08.018905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:22:08.022762 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:08.022739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:22:08.034794 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:08.034767    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 16:22:12.597814 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.597779    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"]
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598066    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="util"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598078    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="util"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598092    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="extract"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598097    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="extract"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598106    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="pull"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598111    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="pull"
Sep 10 16:22:12.600149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.598178    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="37a59b9d-eb29-4a1f-8adc-59a4d8093ffd" containerName="extract"
Sep 10 16:22:12.601110 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.601095    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.604191 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.604168    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skc8x\""
Sep 10 16:22:12.604335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.604175    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 16:22:12.605574 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.605558    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 16:22:12.620583 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.620554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"]
Sep 10 16:22:12.750336 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.750265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.750515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.750398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.750515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.750443    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdfq8\" (UniqueName: \"kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.851044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.850958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdfq8\" (UniqueName: \"kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.851044 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.851004    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.851227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.851065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.851510 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.851489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.851553 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.851502    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.860433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.860405    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdfq8\" (UniqueName: \"kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:12.910376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:12.910340    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:13.189672 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:13.189647    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"]
Sep 10 16:22:13.191815 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:13.191779    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c7e4905_3a79_4770_b9d5_4d0c3fd2c23a.slice/crio-642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff WatchSource:0}: Error finding container 642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff: Status 404 returned error can't find the container with id 642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff
Sep 10 16:22:13.533947 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:13.533852    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerID="9a1391fc2ae9b0309f7401bb6bf8524168d406b6b989d8a3a57b8a0bf025c12f" exitCode=0
Sep 10 16:22:13.533947 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:13.533938    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69" event={"ID":"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a","Type":"ContainerDied","Data":"9a1391fc2ae9b0309f7401bb6bf8524168d406b6b989d8a3a57b8a0bf025c12f"}
Sep 10 16:22:13.534180 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:13.533985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69" event={"ID":"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a","Type":"ContainerStarted","Data":"642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff"}
Sep 10 16:22:13.534439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:13.534424    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:22:14.783960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.783920    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"]
Sep 10 16:22:14.786121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.786098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.789140 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.789111    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 16:22:14.789247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.789174    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-tbzcd\""
Sep 10 16:22:14.789247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.789196    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 16:22:14.789247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.789184    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 16:22:14.789523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.789506    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 16:22:14.805726 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.805700    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"]
Sep 10 16:22:14.869697 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.869669    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-tmp\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.869850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.869706    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdkjq\" (UniqueName: \"kubernetes.io/projected/350e4f97-8b25-4e12-87cb-76f4da7476aa-kube-api-access-jdkjq\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.869850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.869732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.869850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.869812    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-manifests\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.869850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.869840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-manifests\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-tmp\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdkjq\" (UniqueName: \"kubernetes.io/projected/350e4f97-8b25-4e12-87cb-76f4da7476aa-kube-api-access-jdkjq\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.970978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.970956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-manifests\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.973040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.973009    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/350e4f97-8b25-4e12-87cb-76f4da7476aa-tmp\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.973217 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.973196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.973258 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.973196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/350e4f97-8b25-4e12-87cb-76f4da7476aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:14.982484 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:14.982463    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdkjq\" (UniqueName: \"kubernetes.io/projected/350e4f97-8b25-4e12-87cb-76f4da7476aa-kube-api-access-jdkjq\") pod \"opendatahub-operator-controller-manager-57b77968d9-qr82g\" (UID: \"350e4f97-8b25-4e12-87cb-76f4da7476aa\") " pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:15.095636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:15.095602    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:15.378920 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:15.378889    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"]
Sep 10 16:22:15.380756 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:15.380731    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350e4f97_8b25_4e12_87cb_76f4da7476aa.slice/crio-79cf6d6b113c782a6e9cd7a80f5d60ce1e5fbc2475dafa589d0990a49b508354 WatchSource:0}: Error finding container 79cf6d6b113c782a6e9cd7a80f5d60ce1e5fbc2475dafa589d0990a49b508354: Status 404 returned error can't find the container with id 79cf6d6b113c782a6e9cd7a80f5d60ce1e5fbc2475dafa589d0990a49b508354
Sep 10 16:22:15.545220 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:15.545138    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g" event={"ID":"350e4f97-8b25-4e12-87cb-76f4da7476aa","Type":"ContainerStarted","Data":"79cf6d6b113c782a6e9cd7a80f5d60ce1e5fbc2475dafa589d0990a49b508354"}
Sep 10 16:22:15.547089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:15.547056    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerID="e968ddb1f8efbfa5e7ba3d6e23b0d33788d2bdb5eeebaef5eff4904511b4fb24" exitCode=0
Sep 10 16:22:15.547089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:15.547089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69" event={"ID":"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a","Type":"ContainerDied","Data":"e968ddb1f8efbfa5e7ba3d6e23b0d33788d2bdb5eeebaef5eff4904511b4fb24"}
Sep 10 16:22:16.555130 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:16.555087    2577 generic.go:356] "Generic (PLEG): container finished" podID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerID="abe0f7459a7b9546e1a897525bfbde4b6e070bea0451ba25824ed9b68fc6ecd4" exitCode=0
Sep 10 16:22:16.555668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:16.555129    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69" event={"ID":"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a","Type":"ContainerDied","Data":"abe0f7459a7b9546e1a897525bfbde4b6e070bea0451ba25824ed9b68fc6ecd4"}
Sep 10 16:22:18.426411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.426380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:18.501223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.501196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util\") pod \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") "
Sep 10 16:22:18.501366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.501237    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdfq8\" (UniqueName: \"kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8\") pod \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") "
Sep 10 16:22:18.501366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.501335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle\") pod \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\" (UID: \"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a\") "
Sep 10 16:22:18.502083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.502053    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle" (OuterVolumeSpecName: "bundle") pod "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" (UID: "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:18.503531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.503505    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8" (OuterVolumeSpecName: "kube-api-access-gdfq8") pod "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" (UID: "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a"). InnerVolumeSpecName "kube-api-access-gdfq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:18.508978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.508944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util" (OuterVolumeSpecName: "util") pod "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" (UID: "3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:18.566827 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.566792    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g" event={"ID":"350e4f97-8b25-4e12-87cb-76f4da7476aa","Type":"ContainerStarted","Data":"801cb55f54583d4a3bce16022986261f7f202e293b6691c063a3a88dd848631d"}
Sep 10 16:22:18.568740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.568715    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69" event={"ID":"3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a","Type":"ContainerDied","Data":"642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff"}
Sep 10 16:22:18.568880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.568754    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="642651f225976bb05a5776217c94af228c17db8e81d7ec50bdd39871e37de2ff"
Sep 10 16:22:18.568880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.568812    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b08vs69"
Sep 10 16:22:18.602508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.602470    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:18.602508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.602502    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:18.602508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:18.602516    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdfq8\" (UniqueName: \"kubernetes.io/projected/3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a-kube-api-access-gdfq8\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:20.579249 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:20.579215    2577 generic.go:356] "Generic (PLEG): container finished" podID="350e4f97-8b25-4e12-87cb-76f4da7476aa" containerID="801cb55f54583d4a3bce16022986261f7f202e293b6691c063a3a88dd848631d" exitCode=0
Sep 10 16:22:20.579630 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:20.579270    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g" event={"ID":"350e4f97-8b25-4e12-87cb-76f4da7476aa","Type":"ContainerDied","Data":"801cb55f54583d4a3bce16022986261f7f202e293b6691c063a3a88dd848631d"}
Sep 10 16:22:21.585583 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:21.585540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g" event={"ID":"350e4f97-8b25-4e12-87cb-76f4da7476aa","Type":"ContainerStarted","Data":"574b9db36f7c3a82b0ea86158ac66c016f6f766c9caf48036b538cd287484a29"}
Sep 10 16:22:21.586025 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:21.585766    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:21.615124 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:21.612719    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g" podStartSLOduration=3.14489297 podStartE2EDuration="7.612698104s" totalImagesPullingTime="3.436698338s" totalInitContainerRuntime="1.031106796s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:22:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:22:21.611144196 +0000 UTC m=+314.012945325" watchObservedRunningTime="2026-09-10 16:22:21.612698104 +0000 UTC m=+314.014499234"
Sep 10 16:22:32.593459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:32.593424    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57b77968d9-qr82g"
Sep 10 16:22:40.154890 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.154861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"]
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155151    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="util"
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155162    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="util"
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155178    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="pull"
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155183    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="pull"
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155197    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="extract"
Sep 10 16:22:40.155243 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155203    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="extract"
Sep 10 16:22:40.155445 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.155251    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c7e4905-3a79-4770-b9d5-4d0c3fd2c23a" containerName="extract"
Sep 10 16:22:40.158513 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.158488    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.161165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.161139    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 16:22:40.161342 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.161197    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-p7nns\""
Sep 10 16:22:40.161457 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.161440    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 16:22:40.169996 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.169973    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"]
Sep 10 16:22:40.287551 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.287517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.287744 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.287592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tmp\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.287744 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.287644    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltdmw\" (UniqueName: \"kubernetes.io/projected/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-kube-api-access-ltdmw\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.388863 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.388828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tmp\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.389055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.388882    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltdmw\" (UniqueName: \"kubernetes.io/projected/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-kube-api-access-ltdmw\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.389055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.388920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.389055 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:22:40.389012    2577 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 10 16:22:40.389228 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:22:40.389073    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs podName:8a6f63ec-9e5a-4c75-af93-00e618ec9a51 nodeName:}" failed. No retries permitted until 2026-09-10 16:22:40.88905728 +0000 UTC m=+333.290858385 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs") pod "kube-auth-proxy-5cfcc4b4c4-5fzmm" (UID: "8a6f63ec-9e5a-4c75-af93-00e618ec9a51") : secret "kube-auth-proxy-tls" not found
Sep 10 16:22:40.391236 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.391211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tmp\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.398475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.398445    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltdmw\" (UniqueName: \"kubernetes.io/projected/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-kube-api-access-ltdmw\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.893568 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.893520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:40.896062 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:40.896042    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a6f63ec-9e5a-4c75-af93-00e618ec9a51-tls-certs\") pod \"kube-auth-proxy-5cfcc4b4c4-5fzmm\" (UID: \"8a6f63ec-9e5a-4c75-af93-00e618ec9a51\") " pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:41.071515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.071473    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"
Sep 10 16:22:41.345913 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.345870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm"]
Sep 10 16:22:41.348525 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:41.348502    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a6f63ec_9e5a_4c75_af93_00e618ec9a51.slice/crio-6074b2a2ce60a86e4ce95333b9d8e172cfb956a0d8a04ec2343df0493e160420 WatchSource:0}: Error finding container 6074b2a2ce60a86e4ce95333b9d8e172cfb956a0d8a04ec2343df0493e160420: Status 404 returned error can't find the container with id 6074b2a2ce60a86e4ce95333b9d8e172cfb956a0d8a04ec2343df0493e160420
Sep 10 16:22:41.620943 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.620901    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:22:41.625569 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.625546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.629151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.629115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 16:22:41.629151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.629136    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6hh52\""
Sep 10 16:22:41.629620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.629601    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 16:22:41.629620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.629612    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 16:22:41.629797 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.629645    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 16:22:41.644488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.644463    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:22:41.678016 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.677981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm" event={"ID":"8a6f63ec-9e5a-4c75-af93-00e618ec9a51","Type":"ContainerStarted","Data":"6074b2a2ce60a86e4ce95333b9d8e172cfb956a0d8a04ec2343df0493e160420"}
Sep 10 16:22:41.700241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700205    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.700633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.700450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftgrs\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.801784 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.801748    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.801784 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.801787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802018 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.801919    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802018 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.801985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802131 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.802019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftgrs\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802131 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.802104    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.802158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.802835 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.802796    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.804408 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.804386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.804508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.804420    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.804508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.804496    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.804604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.804520    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.812187 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.812160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.813232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.813209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftgrs\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs\") pod \"istiod-openshift-gateway-7b7995d5cd-bqzxk\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:41.935728 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:41.935627    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:42.230287 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:42.230193    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:22:42.234032 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:42.234000    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09af208a_05eb_4da2_887c_c148e9115954.slice/crio-dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638 WatchSource:0}: Error finding container dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638: Status 404 returned error can't find the container with id dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638
Sep 10 16:22:42.684264 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:42.684225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" event={"ID":"09af208a-05eb-4da2-887c-c148e9115954","Type":"ContainerStarted","Data":"dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638"}
Sep 10 16:22:44.698398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:44.698322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm" event={"ID":"8a6f63ec-9e5a-4c75-af93-00e618ec9a51","Type":"ContainerStarted","Data":"433dfed7250800ee8dd72cf996fbdf05d54258a46cb0c9c56dcfc22af8e3709d"}
Sep 10 16:22:44.721127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:44.720998    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5cfcc4b4c4-5fzmm" podStartSLOduration=1.7462490640000001 podStartE2EDuration="4.720976712s" totalImagesPullingTime="2.974727648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:22:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:22:44.718166685 +0000 UTC m=+337.119967813" watchObservedRunningTime="2026-09-10 16:22:44.720976712 +0000 UTC m=+337.122777848"
Sep 10 16:22:46.187488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:46.187434    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:22:46.187870 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:46.187515    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:22:46.712323 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:46.712289    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" event={"ID":"09af208a-05eb-4da2-887c-c148e9115954","Type":"ContainerStarted","Data":"847a0a88d7352dc09f540807bfbd1926ea16cf3cabafc5c6ed86310a4e0cf298"}
Sep 10 16:22:46.712514 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:46.712450    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:46.740109 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:46.740061    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" podStartSLOduration=1.789454648 podStartE2EDuration="5.740046805s" totalImagesPullingTime="3.950592157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:22:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:22:46.739546955 +0000 UTC m=+339.141348079" watchObservedRunningTime="2026-09-10 16:22:46.740046805 +0000 UTC m=+339.141847935"
Sep 10 16:22:47.720442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:47.720406    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bqzxk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 16:22:47.720880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:47.720480    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 16:22:50.720052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:50.720021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:56.591046 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.590997    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"]
Sep 10 16:22:56.601758 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.601735    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.616059 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.616030    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"]
Sep 10 16:22:56.619265 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.619250    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.621463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.621442    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-598bt\""
Sep 10 16:22:56.622633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.622614    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 16:22:56.625381 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.625360    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 16:22:56.625494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.625479    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-9rqmv\""
Sep 10 16:22:56.633571 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.633548    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"]
Sep 10 16:22:56.725001 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.724966    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"]
Sep 10 16:22:56.733968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.733933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac4202d6-b576-4a49-80d9-17499f64277a-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.734239 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.734213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.734348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.734329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-425ff\" (UniqueName: \"kubernetes.io/projected/ac4202d6-b576-4a49-80d9-17499f64277a-kube-api-access-425ff\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.734395 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.734355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2b7x\" (UniqueName: \"kubernetes.io/projected/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-kube-api-access-m2b7x\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.835069 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.835029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-425ff\" (UniqueName: \"kubernetes.io/projected/ac4202d6-b576-4a49-80d9-17499f64277a-kube-api-access-425ff\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.835069 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.835068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2b7x\" (UniqueName: \"kubernetes.io/projected/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-kube-api-access-m2b7x\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.835338 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.835092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac4202d6-b576-4a49-80d9-17499f64277a-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.835338 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.835159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.835338 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:22:56.835291    2577 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 16:22:56.835501 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:22:56.835360    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert podName:cb4182c8-308d-4eec-b0f4-b25f3a998ca0 nodeName:}" failed. No retries permitted until 2026-09-10 16:22:57.335340707 +0000 UTC m=+349.737141826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert") pod "llmisvc-controller-manager-777c6bd7d5-mz45n" (UID: "cb4182c8-308d-4eec-b0f4-b25f3a998ca0") : secret "llmisvc-webhook-server-cert" not found
Sep 10 16:22:56.837635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.837611    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac4202d6-b576-4a49-80d9-17499f64277a-cert\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.846865 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.846814    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-425ff\" (UniqueName: \"kubernetes.io/projected/ac4202d6-b576-4a49-80d9-17499f64277a-kube-api-access-425ff\") pod \"kserve-controller-manager-d6f4b65b8-c6rq9\" (UID: \"ac4202d6-b576-4a49-80d9-17499f64277a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:56.853071 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.853047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2b7x\" (UniqueName: \"kubernetes.io/projected/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-kube-api-access-m2b7x\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:56.911212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:56.911156    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:57.201409 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.201378    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"]
Sep 10 16:22:57.202440 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:57.202411    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac4202d6_b576_4a49_80d9_17499f64277a.slice/crio-1c46f532d75704898151f5558224785df35669f777a73d3a83f2423e6c62eaec WatchSource:0}: Error finding container 1c46f532d75704898151f5558224785df35669f777a73d3a83f2423e6c62eaec: Status 404 returned error can't find the container with id 1c46f532d75704898151f5558224785df35669f777a73d3a83f2423e6c62eaec
Sep 10 16:22:57.340550 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.340491    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:57.342923 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.342899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cb4182c8-308d-4eec-b0f4-b25f3a998ca0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mz45n\" (UID: \"cb4182c8-308d-4eec-b0f4-b25f3a998ca0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:57.528576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.528484    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:22:57.765329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.765296    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9" event={"ID":"ac4202d6-b576-4a49-80d9-17499f64277a","Type":"ContainerStarted","Data":"1c46f532d75704898151f5558224785df35669f777a73d3a83f2423e6c62eaec"}
Sep 10 16:22:57.814569 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:57.814543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"]
Sep 10 16:22:57.816257 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:22:57.816225    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcb4182c8_308d_4eec_b0f4_b25f3a998ca0.slice/crio-055a7a889804507032db3b5da6e770d5cdf8d73e5f391efe289d1b33a7431902 WatchSource:0}: Error finding container 055a7a889804507032db3b5da6e770d5cdf8d73e5f391efe289d1b33a7431902: Status 404 returned error can't find the container with id 055a7a889804507032db3b5da6e770d5cdf8d73e5f391efe289d1b33a7431902
Sep 10 16:22:58.772975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:58.772929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n" event={"ID":"cb4182c8-308d-4eec-b0f4-b25f3a998ca0","Type":"ContainerStarted","Data":"055a7a889804507032db3b5da6e770d5cdf8d73e5f391efe289d1b33a7431902"}
Sep 10 16:22:59.427925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.427892    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:22:59.428142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.428114    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery" containerID="cri-o://847a0a88d7352dc09f540807bfbd1926ea16cf3cabafc5c6ed86310a4e0cf298" gracePeriod=30
Sep 10 16:22:59.781611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.781501    2577 generic.go:356] "Generic (PLEG): container finished" podID="09af208a-05eb-4da2-887c-c148e9115954" containerID="847a0a88d7352dc09f540807bfbd1926ea16cf3cabafc5c6ed86310a4e0cf298" exitCode=0
Sep 10 16:22:59.781611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.781554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" event={"ID":"09af208a-05eb-4da2-887c-c148e9115954","Type":"ContainerDied","Data":"847a0a88d7352dc09f540807bfbd1926ea16cf3cabafc5c6ed86310a4e0cf298"}
Sep 10 16:22:59.781611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.781602    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" event={"ID":"09af208a-05eb-4da2-887c-c148e9115954","Type":"ContainerDied","Data":"dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638"}
Sep 10 16:22:59.782375 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.781622    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dcf70fb09b095c89a1b7fd58d6780270111d6298dd89cf90a20e24df47288638"
Sep 10 16:22:59.784220 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.784193    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9" event={"ID":"ac4202d6-b576-4a49-80d9-17499f64277a","Type":"ContainerStarted","Data":"cf989b6f8c4ce53fd9a1fe2eb00a4bfde7a715d29659034bb2cbb9c60b611065"}
Sep 10 16:22:59.784369 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.784331    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:22:59.788263 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.788241    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:22:59.807090 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.807042    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9" podStartSLOduration=1.743492497 podStartE2EDuration="3.807023916s" totalImagesPullingTime="2.063531419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:22:59.806755045 +0000 UTC m=+352.208556173" watchObservedRunningTime="2026-09-10 16:22:59.807023916 +0000 UTC m=+352.208825044"
Sep 10 16:22:59.861823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.861791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862003 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.861857    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftgrs\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862003 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.861910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862003 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.861960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.862001    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.862032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.862163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.862062    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap\") pod \"09af208a-05eb-4da2-887c-c148e9115954\" (UID: \"09af208a-05eb-4da2-887c-c148e9115954\") "
Sep 10 16:22:59.864093 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.863751    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:22:59.868520 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.867028    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs" (OuterVolumeSpecName: "local-certs") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:22:59.870185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.869822    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:59.870185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.869855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs" (OuterVolumeSpecName: "kube-api-access-ftgrs") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "kube-api-access-ftgrs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:59.870185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.870023    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts" (OuterVolumeSpecName: "cacerts") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:59.870620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.870574    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:22:59.872774 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.871763    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token" (OuterVolumeSpecName: "istio-token") pod "09af208a-05eb-4da2-887c-c148e9115954" (UID: "09af208a-05eb-4da2-887c-c148e9115954"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:22:59.963394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963356    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-cacerts\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963387    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-dns-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963396    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963405    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/09af208a-05eb-4da2-887c-c148e9115954-istio-csr-ca-configmap\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963416    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/09af208a-05eb-4da2-887c-c148e9115954-local-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963424    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftgrs\" (UniqueName: \"kubernetes.io/projected/09af208a-05eb-4da2-887c-c148e9115954-kube-api-access-ftgrs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:22:59.963631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:22:59.963432    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/09af208a-05eb-4da2-887c-c148e9115954-istio-kubeconfig\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:00.788760 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:00.788721    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"
Sep 10 16:23:00.818635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:00.818603    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:23:00.822019 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:00.821996    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk"]
Sep 10 16:23:01.795672 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:01.795586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n" event={"ID":"cb4182c8-308d-4eec-b0f4-b25f3a998ca0","Type":"ContainerStarted","Data":"26fa48b94d2316e778fd4eb63d0bd65b6de0d8020ba4368bd3d6f209ece6beb8"}
Sep 10 16:23:01.796016 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:01.795680    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:23:01.822014 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:01.821961    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n" podStartSLOduration=2.137190751 podStartE2EDuration="5.82194398s" totalImagesPullingTime="3.684753229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:22:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:01.818738007 +0000 UTC m=+354.220539148" watchObservedRunningTime="2026-09-10 16:23:01.82194398 +0000 UTC m=+354.223745146"
Sep 10 16:23:02.111980 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:02.111898    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09af208a-05eb-4da2-887c-c148e9115954" path="/var/lib/kubelet/pods/09af208a-05eb-4da2-887c-c148e9115954/volumes"
Sep 10 16:23:04.719656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:04.719601    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-bqzxk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.34:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 16:23:04.720120 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:04.719685    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-bqzxk" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery" probeResult="failure" output="Get \"http://10.132.0.34:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 16:23:28.677656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.677620    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78956c4984-l4qmm"]
Sep 10 16:23:28.678252 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.678031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery"
Sep 10 16:23:28.678252 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.678048    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery"
Sep 10 16:23:28.678252 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.678119    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="09af208a-05eb-4da2-887c-c148e9115954" containerName="discovery"
Sep 10 16:23:28.690727 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.690689    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.695164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.695136    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78956c4984-l4qmm"]
Sep 10 16:23:28.711444 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q87td\" (UniqueName: \"kubernetes.io/projected/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-kube-api-access-q87td\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711454    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711477    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-service-ca\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-oauth-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711589    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-trusted-ca-bundle\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.711663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.711612    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-oauth-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812088 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q87td\" (UniqueName: \"kubernetes.io/projected/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-kube-api-access-q87td\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812088 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812139    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-service-ca\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812157    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-oauth-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-trusted-ca-bundle\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.812392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.812219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-oauth-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.813183 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.813085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-service-ca\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.813183 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.813117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-oauth-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.813402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.813201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-trusted-ca-bundle\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.813402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.813271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.814981 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.814958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-serving-cert\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.815071 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.815000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-console-oauth-config\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:28.820840 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:28.820820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q87td\" (UniqueName: \"kubernetes.io/projected/97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1-kube-api-access-q87td\") pod \"console-78956c4984-l4qmm\" (UID: \"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1\") " pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:29.001611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:29.001513    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:29.279486 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:29.279432    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78956c4984-l4qmm"]
Sep 10 16:23:29.281793 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:23:29.281752    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97654c5d_1dbf_42e9_bbbe_d5fa52cff4a1.slice/crio-4021a7368acdcf916b7da2eb5a1b02993c31738181eb4442073488962dd09fd7 WatchSource:0}: Error finding container 4021a7368acdcf916b7da2eb5a1b02993c31738181eb4442073488962dd09fd7: Status 404 returned error can't find the container with id 4021a7368acdcf916b7da2eb5a1b02993c31738181eb4442073488962dd09fd7
Sep 10 16:23:29.936124 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:29.936088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78956c4984-l4qmm" event={"ID":"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1","Type":"ContainerStarted","Data":"dae9b1f9d60d8028cdc396ffbde42b203d33640fc67042034d784ab980b895c2"}
Sep 10 16:23:29.936531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:29.936159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78956c4984-l4qmm" event={"ID":"97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1","Type":"ContainerStarted","Data":"4021a7368acdcf916b7da2eb5a1b02993c31738181eb4442073488962dd09fd7"}
Sep 10 16:23:29.957234 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:29.957188    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78956c4984-l4qmm" podStartSLOduration=1.9571760550000001 podStartE2EDuration="1.957176055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:23:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:23:29.956683233 +0000 UTC m=+382.358484360" watchObservedRunningTime="2026-09-10 16:23:29.957176055 +0000 UTC m=+382.358977183"
Sep 10 16:23:30.794400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:30.794370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c6rq9"
Sep 10 16:23:32.802839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:32.802808    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mz45n"
Sep 10 16:23:39.002183 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:39.002150    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:39.002598 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:39.002436    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:39.006648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:39.006628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:39.986667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:39.986638    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78956c4984-l4qmm"
Sep 10 16:23:40.050600 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:40.050557    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:23:51.620011 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.619974    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"]
Sep 10 16:23:51.623797 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.623773    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.627007 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.626987    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 10 16:23:51.627127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.627095    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-skc8x\""
Sep 10 16:23:51.627184 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.627104    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 10 16:23:51.633446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.633421    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"]
Sep 10 16:23:51.692740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.692704    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.692943 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.692756    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.692943 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.692778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mntks\" (UniqueName: \"kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.760736 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.760699    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"]
Sep 10 16:23:51.763471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.763449    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.793218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793190    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"]
Sep 10 16:23:51.793218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.793469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.793469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.793469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793355    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mntks\" (UniqueName: \"kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.793469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.793663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793476    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-797xt\" (UniqueName: \"kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.793722 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.793776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.793743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.812512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.812478    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mntks\" (UniqueName: \"kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.894959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.894855    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.894959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.894923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-797xt\" (UniqueName: \"kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.895189 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.894978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.895379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.895356    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.895450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.895375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.897013 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.896992    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"]
Sep 10 16:23:51.899950 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.899935    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:51.914925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.914889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-797xt\" (UniqueName: \"kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:51.915566 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.915547    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"]
Sep 10 16:23:51.934423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.934389    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:51.976570 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.976536    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"]
Sep 10 16:23:51.979698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.979680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:51.992662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.992592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"]
Sep 10 16:23:51.996014 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.995989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bf55\" (UniqueName: \"kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:51.996132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.996032    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:51.996132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:51.996080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.072020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.071983    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:52.096932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.096897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.097104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.096945    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.097170 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bf55\" (UniqueName: \"kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.097170 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097157    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5mxq\" (UniqueName: \"kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.097313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.097313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.097421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.097509 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.097493    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.107810 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.107775    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bf55\" (UniqueName: \"kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.198654 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.198570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5mxq\" (UniqueName: \"kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.198654 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.198620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.198654 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.198651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.199002 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.198983    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.199068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.199025    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.208593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.208567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5mxq\" (UniqueName: \"kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.209287 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.209259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:52.213045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.212917    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"]
Sep 10 16:23:52.214724 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:23:52.214694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a682217_aab0_492c_988c_b4f1ee05f3a4.slice/crio-94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b WatchSource:0}: Error finding container 94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b: Status 404 returned error can't find the container with id 94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b
Sep 10 16:23:52.296743 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.296718    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:52.346981 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.346913    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"]
Sep 10 16:23:52.389190 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:23:52.389158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod534ea626_0afb_4f49_9097_46367cfa074f.slice/crio-f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52 WatchSource:0}: Error finding container f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52: Status 404 returned error can't find the container with id f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52
Sep 10 16:23:52.494379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.494347    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"]
Sep 10 16:23:52.497241 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:23:52.497194    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod991c41e0_fdf5_4ee2_948d_3b10c90bc1e5.slice/crio-a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531 WatchSource:0}: Error finding container a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531: Status 404 returned error can't find the container with id a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531
Sep 10 16:23:52.576458 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:52.576378    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"]
Sep 10 16:23:52.580311 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:23:52.580269    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod792ce746_b132_4f4c_b24b_73ea819dff63.slice/crio-316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a WatchSource:0}: Error finding container 316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a: Status 404 returned error can't find the container with id 316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a
Sep 10 16:23:53.050648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.050616    2577 generic.go:356] "Generic (PLEG): container finished" podID="534ea626-0afb-4f49-9097-46367cfa074f" containerID="f8207a21506fc19c43a635fb201ce43e5d1dbb66e7b1994902f4376e7c1470e7" exitCode=0
Sep 10 16:23:53.051092 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.050689    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerDied","Data":"f8207a21506fc19c43a635fb201ce43e5d1dbb66e7b1994902f4376e7c1470e7"}
Sep 10 16:23:53.051092 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.050729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerStarted","Data":"f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52"}
Sep 10 16:23:53.052773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.052589    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerID="99eedfe26e4a2d49eb9c037a9e6468b59d1df0d04fbbf47abea843df384f6c56" exitCode=0
Sep 10 16:23:53.052773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.052629    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerDied","Data":"99eedfe26e4a2d49eb9c037a9e6468b59d1df0d04fbbf47abea843df384f6c56"}
Sep 10 16:23:53.052773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.052684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerStarted","Data":"94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b"}
Sep 10 16:23:53.054563 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.054541    2577 generic.go:356] "Generic (PLEG): container finished" podID="792ce746-b132-4f4c-b24b-73ea819dff63" containerID="aae90c662ce82605c10287d57899159acc9f0232318146db22fba1b67b3e8daf" exitCode=0
Sep 10 16:23:53.054654 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.054626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp" event={"ID":"792ce746-b132-4f4c-b24b-73ea819dff63","Type":"ContainerDied","Data":"aae90c662ce82605c10287d57899159acc9f0232318146db22fba1b67b3e8daf"}
Sep 10 16:23:53.054715 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.054685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp" event={"ID":"792ce746-b132-4f4c-b24b-73ea819dff63","Type":"ContainerStarted","Data":"316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a"}
Sep 10 16:23:53.056402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.056380    2577 generic.go:356] "Generic (PLEG): container finished" podID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerID="b355bd0cb9f3cf00073bdde035b6c6341fffe7acfd2ca933aca4a99814ebd946" exitCode=0
Sep 10 16:23:53.056474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.056423    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerDied","Data":"b355bd0cb9f3cf00073bdde035b6c6341fffe7acfd2ca933aca4a99814ebd946"}
Sep 10 16:23:53.056474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:53.056454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerStarted","Data":"a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531"}
Sep 10 16:23:54.070691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:54.070618    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerStarted","Data":"749a20052109b6acf90f8fe4ce897c34c1e3491f49ab6d6751a9f9beb1b17caf"}
Sep 10 16:23:54.074093 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:54.074057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerStarted","Data":"b5681d9044524d2db07c637245af95b4f80f7c8826bcdea7be1f1554ec576c06"}
Sep 10 16:23:54.077609 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:54.077582    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerStarted","Data":"bee88e64582401806889a0b7a52c0d59ff62662ff61e1c2f5867e6833ad5975c"}
Sep 10 16:23:55.084593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.084555    2577 generic.go:356] "Generic (PLEG): container finished" podID="792ce746-b132-4f4c-b24b-73ea819dff63" containerID="a2ec36f94a87225d519d55438c7b16fe26177859250dc8c535bc5597f49b4f35" exitCode=0
Sep 10 16:23:55.085004 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.084634    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp" event={"ID":"792ce746-b132-4f4c-b24b-73ea819dff63","Type":"ContainerDied","Data":"a2ec36f94a87225d519d55438c7b16fe26177859250dc8c535bc5597f49b4f35"}
Sep 10 16:23:55.086467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.086446    2577 generic.go:356] "Generic (PLEG): container finished" podID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerID="749a20052109b6acf90f8fe4ce897c34c1e3491f49ab6d6751a9f9beb1b17caf" exitCode=0
Sep 10 16:23:55.086575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.086524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerDied","Data":"749a20052109b6acf90f8fe4ce897c34c1e3491f49ab6d6751a9f9beb1b17caf"}
Sep 10 16:23:55.088367 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.088345    2577 generic.go:356] "Generic (PLEG): container finished" podID="534ea626-0afb-4f49-9097-46367cfa074f" containerID="b5681d9044524d2db07c637245af95b4f80f7c8826bcdea7be1f1554ec576c06" exitCode=0
Sep 10 16:23:55.088463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.088419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerDied","Data":"b5681d9044524d2db07c637245af95b4f80f7c8826bcdea7be1f1554ec576c06"}
Sep 10 16:23:55.090385 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.090363    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerID="bee88e64582401806889a0b7a52c0d59ff62662ff61e1c2f5867e6833ad5975c" exitCode=0
Sep 10 16:23:55.090461 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:55.090443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerDied","Data":"bee88e64582401806889a0b7a52c0d59ff62662ff61e1c2f5867e6833ad5975c"}
Sep 10 16:23:56.097902 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.097868    2577 generic.go:356] "Generic (PLEG): container finished" podID="792ce746-b132-4f4c-b24b-73ea819dff63" containerID="b70234c24a14a7d42ca035d873b7323cd8536d343a2f44b32f51087a9d1fa718" exitCode=0
Sep 10 16:23:56.098353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.097964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp" event={"ID":"792ce746-b132-4f4c-b24b-73ea819dff63","Type":"ContainerDied","Data":"b70234c24a14a7d42ca035d873b7323cd8536d343a2f44b32f51087a9d1fa718"}
Sep 10 16:23:56.099983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.099958    2577 generic.go:356] "Generic (PLEG): container finished" podID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerID="12a6eae983d1041fc183ffc1b842c2efc34c03f035eee0b07fc5df25b7ac700e" exitCode=0
Sep 10 16:23:56.100083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.100037    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerDied","Data":"12a6eae983d1041fc183ffc1b842c2efc34c03f035eee0b07fc5df25b7ac700e"}
Sep 10 16:23:56.101939 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.101921    2577 generic.go:356] "Generic (PLEG): container finished" podID="534ea626-0afb-4f49-9097-46367cfa074f" containerID="fec590b1cb394f9a7adb7b35e8346d4d2c8a24c39337e9226305f9d1cc1a0048" exitCode=0
Sep 10 16:23:56.102036 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.101976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerDied","Data":"fec590b1cb394f9a7adb7b35e8346d4d2c8a24c39337e9226305f9d1cc1a0048"}
Sep 10 16:23:56.103946 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.103925    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerID="1306923d0da73e15c469ee3672fb00f30468ae9f4a2c557e41f4c3ed69903289" exitCode=0
Sep 10 16:23:56.104039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:56.104015    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerDied","Data":"1306923d0da73e15c469ee3672fb00f30468ae9f4a2c557e41f4c3ed69903289"}
Sep 10 16:23:57.283846 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.283820    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:57.285911 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.285890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:57.329695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.329666    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:57.333670 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.333640    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:57.343262 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343238    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util\") pod \"534ea626-0afb-4f49-9097-46367cfa074f\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") "
Sep 10 16:23:57.343407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343303    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle\") pod \"534ea626-0afb-4f49-9097-46367cfa074f\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") "
Sep 10 16:23:57.343407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-797xt\" (UniqueName: \"kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt\") pod \"534ea626-0afb-4f49-9097-46367cfa074f\" (UID: \"534ea626-0afb-4f49-9097-46367cfa074f\") "
Sep 10 16:23:57.343407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343363    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bf55\" (UniqueName: \"kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55\") pod \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") "
Sep 10 16:23:57.343407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343386    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle\") pod \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") "
Sep 10 16:23:57.343604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util\") pod \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\" (UID: \"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5\") "
Sep 10 16:23:57.344024 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.343975    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle" (OuterVolumeSpecName: "bundle") pod "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" (UID: "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.344166 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.344116    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle" (OuterVolumeSpecName: "bundle") pod "534ea626-0afb-4f49-9097-46367cfa074f" (UID: "534ea626-0afb-4f49-9097-46367cfa074f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.345668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.345639    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55" (OuterVolumeSpecName: "kube-api-access-9bf55") pod "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" (UID: "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5"). InnerVolumeSpecName "kube-api-access-9bf55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:57.345888 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.345865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt" (OuterVolumeSpecName: "kube-api-access-797xt") pod "534ea626-0afb-4f49-9097-46367cfa074f" (UID: "534ea626-0afb-4f49-9097-46367cfa074f"). InnerVolumeSpecName "kube-api-access-797xt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:57.349076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.349051    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util" (OuterVolumeSpecName: "util") pod "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" (UID: "991c41e0-fdf5-4ee2-948d-3b10c90bc1e5"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.349261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.349239    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util" (OuterVolumeSpecName: "util") pod "534ea626-0afb-4f49-9097-46367cfa074f" (UID: "534ea626-0afb-4f49-9097-46367cfa074f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.444744 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444700    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util\") pod \"792ce746-b132-4f4c-b24b-73ea819dff63\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") "
Sep 10 16:23:57.444744 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle\") pod \"9a682217-aab0-492c-988c-b4f1ee05f3a4\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") "
Sep 10 16:23:57.444991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444772    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle\") pod \"792ce746-b132-4f4c-b24b-73ea819dff63\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") "
Sep 10 16:23:57.444991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444793    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mntks\" (UniqueName: \"kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks\") pod \"9a682217-aab0-492c-988c-b4f1ee05f3a4\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") "
Sep 10 16:23:57.444991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5mxq\" (UniqueName: \"kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq\") pod \"792ce746-b132-4f4c-b24b-73ea819dff63\" (UID: \"792ce746-b132-4f4c-b24b-73ea819dff63\") "
Sep 10 16:23:57.444991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.444865    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util\") pod \"9a682217-aab0-492c-988c-b4f1ee05f3a4\" (UID: \"9a682217-aab0-492c-988c-b4f1ee05f3a4\") "
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445023    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445040    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445053    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/534ea626-0afb-4f49-9097-46367cfa074f-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445066    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-797xt\" (UniqueName: \"kubernetes.io/projected/534ea626-0afb-4f49-9097-46367cfa074f-kube-api-access-797xt\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445080    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bf55\" (UniqueName: \"kubernetes.io/projected/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-kube-api-access-9bf55\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445094    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/991c41e0-fdf5-4ee2-948d-3b10c90bc1e5-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.445486 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445353    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle" (OuterVolumeSpecName: "bundle") pod "9a682217-aab0-492c-988c-b4f1ee05f3a4" (UID: "9a682217-aab0-492c-988c-b4f1ee05f3a4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.445677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.445652    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle" (OuterVolumeSpecName: "bundle") pod "792ce746-b132-4f4c-b24b-73ea819dff63" (UID: "792ce746-b132-4f4c-b24b-73ea819dff63"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.447032 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.447007    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks" (OuterVolumeSpecName: "kube-api-access-mntks") pod "9a682217-aab0-492c-988c-b4f1ee05f3a4" (UID: "9a682217-aab0-492c-988c-b4f1ee05f3a4"). InnerVolumeSpecName "kube-api-access-mntks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:57.447396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.447376    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq" (OuterVolumeSpecName: "kube-api-access-b5mxq") pod "792ce746-b132-4f4c-b24b-73ea819dff63" (UID: "792ce746-b132-4f4c-b24b-73ea819dff63"). InnerVolumeSpecName "kube-api-access-b5mxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:23:57.450799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.450767    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util" (OuterVolumeSpecName: "util") pod "792ce746-b132-4f4c-b24b-73ea819dff63" (UID: "792ce746-b132-4f4c-b24b-73ea819dff63"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.451357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.451336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util" (OuterVolumeSpecName: "util") pod "9a682217-aab0-492c-988c-b4f1ee05f3a4" (UID: "9a682217-aab0-492c-988c-b4f1ee05f3a4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:23:57.546270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546228    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.546270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546264    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.546270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546272    2577 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/792ce746-b132-4f4c-b24b-73ea819dff63-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.546270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546296    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mntks\" (UniqueName: \"kubernetes.io/projected/9a682217-aab0-492c-988c-b4f1ee05f3a4-kube-api-access-mntks\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.546538 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546306    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5mxq\" (UniqueName: \"kubernetes.io/projected/792ce746-b132-4f4c-b24b-73ea819dff63-kube-api-access-b5mxq\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:57.546538 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:57.546315    2577 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9a682217-aab0-492c-988c-b4f1ee05f3a4-util\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:23:58.118247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.118216    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp"
Sep 10 16:23:58.118452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.118225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847amx9wp" event={"ID":"792ce746-b132-4f4c-b24b-73ea819dff63","Type":"ContainerDied","Data":"316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a"}
Sep 10 16:23:58.118452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.118340    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="316e4c113d2c3dfc07e5e4ea8e9787e5241cc7d3bec7ad17bf7a2282b8719a1a"
Sep 10 16:23:58.121932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.121776    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km"
Sep 10 16:23:58.121932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.121785    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7lp6km" event={"ID":"991c41e0-fdf5-4ee2-948d-3b10c90bc1e5","Type":"ContainerDied","Data":"a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531"}
Sep 10 16:23:58.121932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.121823    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9af32f7b6033a072c8eb4a7d63437eb7f83b29497a62c23b4ff45d1241a2531"
Sep 10 16:23:58.124072 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.124056    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22"
Sep 10 16:23:58.124254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.124051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c9wv22" event={"ID":"534ea626-0afb-4f49-9097-46367cfa074f","Type":"ContainerDied","Data":"f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52"}
Sep 10 16:23:58.124366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.124294    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f0906ff6736f429968ec66073304df71ce01edf411c6d105e82f63b577788d52"
Sep 10 16:23:58.126614 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.126588    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt" event={"ID":"9a682217-aab0-492c-988c-b4f1ee05f3a4","Type":"ContainerDied","Data":"94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b"}
Sep 10 16:23:58.126685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.126620    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c66cmvt"
Sep 10 16:23:58.126685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:23:58.126628    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="94d30937acb17d80a0df8829705a4231ade291baf2f42957f71a784ba1b7175b"
Sep 10 16:24:04.139224 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139189    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"]
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139516    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139528    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139540    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139546    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139555    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139561    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139569    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139574    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139582    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139587    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139593    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139597    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139602    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139607    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139616    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139620    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139625    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139630    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="pull"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139635    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139640    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="extract"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139648    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139653    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139659    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="util"
Sep 10 16:24:04.139699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139663    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="util"
Sep 10 16:24:04.140414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139715    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="991c41e0-fdf5-4ee2-948d-3b10c90bc1e5" containerName="extract"
Sep 10 16:24:04.140414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139723    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="792ce746-b132-4f4c-b24b-73ea819dff63" containerName="extract"
Sep 10 16:24:04.140414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139729    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="534ea626-0afb-4f49-9097-46367cfa074f" containerName="extract"
Sep 10 16:24:04.140414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.139736    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a682217-aab0-492c-988c-b4f1ee05f3a4" containerName="extract"
Sep 10 16:24:04.142685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.142669    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:04.148455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.148430    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 16:24:04.148455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.148450    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 16:24:04.148670 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.148454    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-dfkxs\""
Sep 10 16:24:04.149599 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.149583    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 16:24:04.166510 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.166475    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"]
Sep 10 16:24:04.198668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.198628    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs8jq\" (UniqueName: \"kubernetes.io/projected/fdaeb9a9-0402-4834-a07e-bb0352992943-kube-api-access-zs8jq\") pod \"dns-operator-controller-manager-7b84cd7c9c-hqjz4\" (UID: \"fdaeb9a9-0402-4834-a07e-bb0352992943\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:04.299266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.299229    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs8jq\" (UniqueName: \"kubernetes.io/projected/fdaeb9a9-0402-4834-a07e-bb0352992943-kube-api-access-zs8jq\") pod \"dns-operator-controller-manager-7b84cd7c9c-hqjz4\" (UID: \"fdaeb9a9-0402-4834-a07e-bb0352992943\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:04.314498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.314462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs8jq\" (UniqueName: \"kubernetes.io/projected/fdaeb9a9-0402-4834-a07e-bb0352992943-kube-api-access-zs8jq\") pod \"dns-operator-controller-manager-7b84cd7c9c-hqjz4\" (UID: \"fdaeb9a9-0402-4834-a07e-bb0352992943\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:04.452894 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.452808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:04.759167 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:04.759083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"]
Sep 10 16:24:04.762400 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:04.762370    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdaeb9a9_0402_4834_a07e_bb0352992943.slice/crio-c0e2d594c1393d286aaeb94750c2c7bb82737dc7aadd3d9448eb69dadb28be27 WatchSource:0}: Error finding container c0e2d594c1393d286aaeb94750c2c7bb82737dc7aadd3d9448eb69dadb28be27: Status 404 returned error can't find the container with id c0e2d594c1393d286aaeb94750c2c7bb82737dc7aadd3d9448eb69dadb28be27
Sep 10 16:24:05.071211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.071120    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d878897b-bzbl5" podUID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" containerName="console" containerID="cri-o://57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200" gracePeriod=15
Sep 10 16:24:05.164612 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.164573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4" event={"ID":"fdaeb9a9-0402-4834-a07e-bb0352992943","Type":"ContainerStarted","Data":"c0e2d594c1393d286aaeb94750c2c7bb82737dc7aadd3d9448eb69dadb28be27"}
Sep 10 16:24:05.307822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.307792    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:24:05.408190 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408216    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408246    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zthm\" (UniqueName: \"kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408313    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408395    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle\") pod \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\" (UID: \"6a996ff0-2ba4-49b4-af18-146e6dd6980f\") "
Sep 10 16:24:05.408719 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408683    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config" (OuterVolumeSpecName: "console-config") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:24:05.408719 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:24:05.408719 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408709    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:24:05.408940 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.408916    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:24:05.410517 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.410495    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm" (OuterVolumeSpecName: "kube-api-access-7zthm") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "kube-api-access-7zthm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:05.410882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.410865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:24:05.410955 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.410887    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a996ff0-2ba4-49b4-af18-146e6dd6980f" (UID: "6a996ff0-2ba4-49b4-af18-146e6dd6980f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:24:05.509663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509630    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-oauth-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509659    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509671    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-oauth-config\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509683    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zthm\" (UniqueName: \"kubernetes.io/projected/6a996ff0-2ba4-49b4-af18-146e6dd6980f-kube-api-access-7zthm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509696    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509706    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a996ff0-2ba4-49b4-af18-146e6dd6980f-trusted-ca-bundle\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:05.509925 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:05.509717    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a996ff0-2ba4-49b4-af18-146e6dd6980f-console-serving-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:06.040089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.040050    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"]
Sep 10 16:24:06.040468 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.040442    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" containerName="console"
Sep 10 16:24:06.040468 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.040461    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" containerName="console"
Sep 10 16:24:06.040656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.040569    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" containerName="console"
Sep 10 16:24:06.043849 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.043825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:06.047670 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.047649    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-kpz2k\""
Sep 10 16:24:06.068762 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.068727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"]
Sep 10 16:24:06.115322 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.115259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq92d\" (UniqueName: \"kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d\") pod \"limitador-operator-controller-manager-74495c69f8-dl8cw\" (UID: \"f35d8a05-1b60-4287-8714-e4b0da93d0d1\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:06.174189 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.174157    2577 generic.go:356] "Generic (PLEG): container finished" podID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" containerID="57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200" exitCode=0
Sep 10 16:24:06.174772 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.174199    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d878897b-bzbl5"
Sep 10 16:24:06.174772 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.174224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d878897b-bzbl5" event={"ID":"6a996ff0-2ba4-49b4-af18-146e6dd6980f","Type":"ContainerDied","Data":"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"}
Sep 10 16:24:06.174772 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.174265    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d878897b-bzbl5" event={"ID":"6a996ff0-2ba4-49b4-af18-146e6dd6980f","Type":"ContainerDied","Data":"935b6f130abab407f090dcf76f4441870e890060e00b547e309da1d00ce661d8"}
Sep 10 16:24:06.174772 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.174311    2577 scope.go:117] "RemoveContainer" containerID="57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"
Sep 10 16:24:06.184962 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.184914    2577 scope.go:117] "RemoveContainer" containerID="57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"
Sep 10 16:24:06.185395 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:06.185362    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200\": container with ID starting with 57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200 not found: ID does not exist" containerID="57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"
Sep 10 16:24:06.185507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.185405    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200"} err="failed to get container status \"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200\": rpc error: code = NotFound desc = could not find container \"57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200\": container with ID starting with 57744e1d94f5d62d6b2715c3969c7765d63303c1e0727947b73d39abee9f3200 not found: ID does not exist"
Sep 10 16:24:06.205300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.205248    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:24:06.209668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.209640    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d878897b-bzbl5"]
Sep 10 16:24:06.216163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.216132    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq92d\" (UniqueName: \"kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d\") pod \"limitador-operator-controller-manager-74495c69f8-dl8cw\" (UID: \"f35d8a05-1b60-4287-8714-e4b0da93d0d1\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:06.238334 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.238297    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq92d\" (UniqueName: \"kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d\") pod \"limitador-operator-controller-manager-74495c69f8-dl8cw\" (UID: \"f35d8a05-1b60-4287-8714-e4b0da93d0d1\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:06.357201 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:06.357117    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:07.627372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:07.625720    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"]
Sep 10 16:24:08.112527 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:08.112496    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a996ff0-2ba4-49b4-af18-146e6dd6980f" path="/var/lib/kubelet/pods/6a996ff0-2ba4-49b4-af18-146e6dd6980f/volumes"
Sep 10 16:24:08.188758 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:08.188709    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4" event={"ID":"fdaeb9a9-0402-4834-a07e-bb0352992943","Type":"ContainerStarted","Data":"0eccbb2c59d49b359610293b928505f4cab9920dc9e499723a2cbe00dfd6a9e5"}
Sep 10 16:24:08.188948 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:08.188839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:08.199259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:08.199225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw" event={"ID":"f35d8a05-1b60-4287-8714-e4b0da93d0d1","Type":"ContainerStarted","Data":"247194447c59df989dae579657e9211427dfa72d8fe0288a2cb111ba82e73911"}
Sep 10 16:24:08.223192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:08.223114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4" podStartSLOduration=1.570751042 podStartE2EDuration="4.223097271s" totalImagesPullingTime="2.652346229s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:08.221924191 +0000 UTC m=+420.623725312" watchObservedRunningTime="2026-09-10 16:24:08.223097271 +0000 UTC m=+420.624898399"
Sep 10 16:24:10.212270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.212229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw" event={"ID":"f35d8a05-1b60-4287-8714-e4b0da93d0d1","Type":"ContainerStarted","Data":"faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69"}
Sep 10 16:24:10.212679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.212395    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:10.234873 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.234810    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw" podStartSLOduration=2.068679966 podStartE2EDuration="4.23479474s" totalImagesPullingTime="2.166114774s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:10.23246714 +0000 UTC m=+422.634268268" watchObservedRunningTime="2026-09-10 16:24:10.23479474 +0000 UTC m=+422.636595867"
Sep 10 16:24:10.767055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.767020    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"]
Sep 10 16:24:10.770437 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.770421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.773818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.773797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sq4hz\""
Sep 10 16:24:10.785300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.785258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"]
Sep 10 16:24:10.857026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.856983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.857026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.857029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh2j6\" (UniqueName: \"kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.958486 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.958444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.958683 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.958495    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh2j6\" (UniqueName: \"kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.958878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.958846    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:10.972626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:10.972589    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh2j6\" (UniqueName: \"kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6\") pod \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:11.080739 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:11.080642    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:11.389419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:11.389396    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"]
Sep 10 16:24:11.392313 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:11.392267    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd352aaf3_620e_49b9_9be0_d960389b113c.slice/crio-b23dff1122acbf1b6c549977a178256517d11d1e3223cd465507f2cd02df83cf WatchSource:0}: Error finding container b23dff1122acbf1b6c549977a178256517d11d1e3223cd465507f2cd02df83cf: Status 404 returned error can't find the container with id b23dff1122acbf1b6c549977a178256517d11d1e3223cd465507f2cd02df83cf
Sep 10 16:24:12.224691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:12.224654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk" event={"ID":"d352aaf3-620e-49b9-9be0-d960389b113c","Type":"ContainerStarted","Data":"b23dff1122acbf1b6c549977a178256517d11d1e3223cd465507f2cd02df83cf"}
Sep 10 16:24:13.733466 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.733427    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tprdg"]
Sep 10 16:24:13.737151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.737128    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:13.742132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.742096    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qp7zb\""
Sep 10 16:24:13.761796 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.761769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tprdg"]
Sep 10 16:24:13.882992 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.882948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnt9\" (UniqueName: \"kubernetes.io/projected/50cd86ce-43c2-484e-92a3-2b746f933843-kube-api-access-nqnt9\") pod \"authorino-operator-76b478bd78-tprdg\" (UID: \"50cd86ce-43c2-484e-92a3-2b746f933843\") " pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:13.984199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.984124    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnt9\" (UniqueName: \"kubernetes.io/projected/50cd86ce-43c2-484e-92a3-2b746f933843-kube-api-access-nqnt9\") pod \"authorino-operator-76b478bd78-tprdg\" (UID: \"50cd86ce-43c2-484e-92a3-2b746f933843\") " pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:13.999802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:13.999776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnt9\" (UniqueName: \"kubernetes.io/projected/50cd86ce-43c2-484e-92a3-2b746f933843-kube-api-access-nqnt9\") pod \"authorino-operator-76b478bd78-tprdg\" (UID: \"50cd86ce-43c2-484e-92a3-2b746f933843\") " pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:14.049707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:14.049670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:15.535578 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:15.535518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-tprdg"]
Sep 10 16:24:15.538983 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:15.538952    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cd86ce_43c2_484e_92a3_2b746f933843.slice/crio-6f331d6e05c320f3755387a74c4160b48e1258bd6312d12cad8a84d986f8bdf7 WatchSource:0}: Error finding container 6f331d6e05c320f3755387a74c4160b48e1258bd6312d12cad8a84d986f8bdf7: Status 404 returned error can't find the container with id 6f331d6e05c320f3755387a74c4160b48e1258bd6312d12cad8a84d986f8bdf7
Sep 10 16:24:16.251917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:16.251866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk" event={"ID":"d352aaf3-620e-49b9-9be0-d960389b113c","Type":"ContainerStarted","Data":"2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03"}
Sep 10 16:24:16.252103 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:16.252052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:16.253240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:16.253220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg" event={"ID":"50cd86ce-43c2-484e-92a3-2b746f933843","Type":"ContainerStarted","Data":"6f331d6e05c320f3755387a74c4160b48e1258bd6312d12cad8a84d986f8bdf7"}
Sep 10 16:24:16.278725 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:16.278677    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk" podStartSLOduration=2.337476919 podStartE2EDuration="6.278661224s" totalImagesPullingTime="3.941184305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:16.27703093 +0000 UTC m=+428.678832059" watchObservedRunningTime="2026-09-10 16:24:16.278661224 +0000 UTC m=+428.680462352"
Sep 10 16:24:17.261303 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:17.261251    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg" event={"ID":"50cd86ce-43c2-484e-92a3-2b746f933843","Type":"ContainerStarted","Data":"81a4a0e3c3908c4d768b5130def51cbeb46a47240b9f1af5de72c5561279d1d8"}
Sep 10 16:24:17.288628 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:17.288558    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg" podStartSLOduration=2.751607114 podStartE2EDuration="4.28853582s" totalImagesPullingTime="1.536928706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:17.287338592 +0000 UTC m=+429.689139721" watchObservedRunningTime="2026-09-10 16:24:17.28853582 +0000 UTC m=+429.690336947"
Sep 10 16:24:18.254765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.254728    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"]
Sep 10 16:24:18.258669 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.258653    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.262134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.262112    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qnplg\""
Sep 10 16:24:18.262536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.262254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 16:24:18.262536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.262260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 16:24:18.267768 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.267746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"]
Sep 10 16:24:18.269094 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.269072    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:18.424122 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.424078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4d06266-939a-46d8-8561-26d17e24b558-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.424356 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.424236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xskc2\" (UniqueName: \"kubernetes.io/projected/a4d06266-939a-46d8-8561-26d17e24b558-kube-api-access-xskc2\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.424356 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.424319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.525829 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.525722    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xskc2\" (UniqueName: \"kubernetes.io/projected/a4d06266-939a-46d8-8561-26d17e24b558-kube-api-access-xskc2\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.525829 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.525772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.525829 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.525822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4d06266-939a-46d8-8561-26d17e24b558-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.526105 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:18.525937    2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 16:24:18.526105 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:18.526020    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert podName:a4d06266-939a-46d8-8561-26d17e24b558 nodeName:}" failed. No retries permitted until 2026-09-10 16:24:19.026002797 +0000 UTC m=+431.427803903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-8jh4k" (UID: "a4d06266-939a-46d8-8561-26d17e24b558") : secret "plugin-serving-cert" not found
Sep 10 16:24:18.526521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.526504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a4d06266-939a-46d8-8561-26d17e24b558-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:18.536249 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:18.536227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xskc2\" (UniqueName: \"kubernetes.io/projected/a4d06266-939a-46d8-8561-26d17e24b558-kube-api-access-xskc2\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:19.029669 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:19.029632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:19.032058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:19.032024    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a4d06266-939a-46d8-8561-26d17e24b558-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-8jh4k\" (UID: \"a4d06266-939a-46d8-8561-26d17e24b558\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:19.169826 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:19.169789    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"
Sep 10 16:24:19.208479 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:19.208446    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-hqjz4"
Sep 10 16:24:19.446851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:19.446826    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k"]
Sep 10 16:24:19.447824 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:19.447794    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4d06266_939a_46d8_8561_26d17e24b558.slice/crio-5ee7cfcc20d03963b93aafeff2827993b6754910db28cb33de731c31e5aaa34a WatchSource:0}: Error finding container 5ee7cfcc20d03963b93aafeff2827993b6754910db28cb33de731c31e5aaa34a: Status 404 returned error can't find the container with id 5ee7cfcc20d03963b93aafeff2827993b6754910db28cb33de731c31e5aaa34a
Sep 10 16:24:20.285092 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:20.285056    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k" event={"ID":"a4d06266-939a-46d8-8561-26d17e24b558","Type":"ContainerStarted","Data":"5ee7cfcc20d03963b93aafeff2827993b6754910db28cb33de731c31e5aaa34a"}
Sep 10 16:24:21.220473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:21.220423    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:24.312963 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:24.312932    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k" event={"ID":"a4d06266-939a-46d8-8561-26d17e24b558","Type":"ContainerStarted","Data":"75cac94fb925d98ca6ddc6e49a65c53149de1bae173f0bd7f5fb0a3cd121426c"}
Sep 10 16:24:24.334121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:24.334073    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-8jh4k" podStartSLOduration=1.56736109 podStartE2EDuration="6.334055681s" totalImagesPullingTime="4.766694591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:24.333873986 +0000 UTC m=+436.735675114" watchObservedRunningTime="2026-09-10 16:24:24.334055681 +0000 UTC m=+436.735856807"
Sep 10 16:24:27.263474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:27.263441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:28.946198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:28.946163    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"]
Sep 10 16:24:28.951165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:28.951144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:28.968713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:28.968686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"]
Sep 10 16:24:29.103038 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.102995    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"]
Sep 10 16:24:29.103337 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.103313    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-p2975], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4" podUID="09823101-0324-4feb-94de-5a5bac71af98"
Sep 10 16:24:29.117069 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.117038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2975\" (UniqueName: \"kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.117199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.117074    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.218115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.218031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2975\" (UniqueName: \"kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.218115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.218070    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.218470 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.218453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.228166 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.228142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2975\" (UniqueName: \"kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975\") pod \"kuadrant-operator-controller-manager-5f685c496f-mt6t4\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.276656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.276628    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-tprdg"
Sep 10 16:24:29.340734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.340705    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.348376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.348289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:29.473179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.473075    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"]
Sep 10 16:24:29.475374 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.475344    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"]
Sep 10 16:24:29.485445 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.485414    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"]
Sep 10 16:24:29.485665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.485639    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" containerName="manager" containerID="cri-o://2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03" gracePeriod=2
Sep 10 16:24:29.494944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.494840    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"]
Sep 10 16:24:29.519976 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.519946    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2975\" (UniqueName: \"kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975\") pod \"09823101-0324-4feb-94de-5a5bac71af98\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") "
Sep 10 16:24:29.519976 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.519980    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume\") pod \"09823101-0324-4feb-94de-5a5bac71af98\" (UID: \"09823101-0324-4feb-94de-5a5bac71af98\") "
Sep 10 16:24:29.520351 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.520322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "09823101-0324-4feb-94de-5a5bac71af98" (UID: "09823101-0324-4feb-94de-5a5bac71af98"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:24:29.522023 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.521999    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975" (OuterVolumeSpecName: "kube-api-access-p2975") pod "09823101-0324-4feb-94de-5a5bac71af98" (UID: "09823101-0324-4feb-94de-5a5bac71af98"). InnerVolumeSpecName "kube-api-access-p2975". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:29.534993 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.534963    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:29.535521 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.535502    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" containerName="manager"
Sep 10 16:24:29.535575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.535526    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" containerName="manager"
Sep 10 16:24:29.535661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.535648    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" containerName="manager"
Sep 10 16:24:29.539259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.539236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.547516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.547495    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"]
Sep 10 16:24:29.547720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.547697    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" containerName="manager" containerID="cri-o://faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69" gracePeriod=2
Sep 10 16:24:29.561506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.561476    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:29.585647 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.585616    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"]
Sep 10 16:24:29.587565 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.587543    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:29.587895 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.587883    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" containerName="manager"
Sep 10 16:24:29.587931 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.587897    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" containerName="manager"
Sep 10 16:24:29.587969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.587958    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" containerName="manager"
Sep 10 16:24:29.591750 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.591729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.605778 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.605739    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.621354 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.621332    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2975\" (UniqueName: \"kubernetes.io/projected/09823101-0324-4feb-94de-5a5bac71af98-kube-api-access-p2975\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:29.621463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.621357    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/09823101-0324-4feb-94de-5a5bac71af98-extensions-socket-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:29.632041 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.632012    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:29.642547 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.642519    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"]
Sep 10 16:24:29.646602 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.646579    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:29.661183 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.661146    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dl8cw\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:29.664396 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.664152    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.666021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.665993    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"]
Sep 10 16:24:29.704596 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.704549    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dl8cw\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:29.707114 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.707090    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.722839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.722806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbm9\" (UniqueName: \"kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.722945 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.722897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxbfn\" (UniqueName: \"kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.723017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.722961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.723017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.722991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.784037 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.784014    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.786935 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.786914    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:29.787122 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.787102    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dl8cw\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:29.789619 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.789596    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.791835 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.791810    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-dl8cw\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:29.794385 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:29.794365    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-bmlzk\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:29.823835 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.823794    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbm9\" (UniqueName: \"kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.823979 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.823842    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl2kw\" (UniqueName: \"kubernetes.io/projected/9a2f4b97-7fb7-4b92-ac37-7666a5f774f4-kube-api-access-kl2kw\") pod \"limitador-operator-controller-manager-74495c69f8-tsxwj\" (UID: \"9a2f4b97-7fb7-4b92-ac37-7666a5f774f4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:29.823979 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.823948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxbfn\" (UniqueName: \"kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.824054 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.823996    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.824054 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.824016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.824449 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.824432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.824491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.824472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.850254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.850225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbm9\" (UniqueName: \"kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-89x94\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.850878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.850859    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxbfn\" (UniqueName: \"kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn\") pod \"kuadrant-operator-controller-manager-76f8f686bf-qwhv9\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.866693 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.866670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:29.921618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.921590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:29.925237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.925146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume\") pod \"d352aaf3-620e-49b9-9be0-d960389b113c\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") "
Sep 10 16:24:29.925237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.925192    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xq92d\" (UniqueName: \"kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d\") pod \"f35d8a05-1b60-4287-8714-e4b0da93d0d1\" (UID: \"f35d8a05-1b60-4287-8714-e4b0da93d0d1\") "
Sep 10 16:24:29.925430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.925251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh2j6\" (UniqueName: \"kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6\") pod \"d352aaf3-620e-49b9-9be0-d960389b113c\" (UID: \"d352aaf3-620e-49b9-9be0-d960389b113c\") "
Sep 10 16:24:29.925516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.925494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl2kw\" (UniqueName: \"kubernetes.io/projected/9a2f4b97-7fb7-4b92-ac37-7666a5f774f4-kube-api-access-kl2kw\") pod \"limitador-operator-controller-manager-74495c69f8-tsxwj\" (UID: \"9a2f4b97-7fb7-4b92-ac37-7666a5f774f4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:29.925837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.925803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d352aaf3-620e-49b9-9be0-d960389b113c" (UID: "d352aaf3-620e-49b9-9be0-d960389b113c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:24:29.930060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.930035    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d" (OuterVolumeSpecName: "kube-api-access-xq92d") pod "f35d8a05-1b60-4287-8714-e4b0da93d0d1" (UID: "f35d8a05-1b60-4287-8714-e4b0da93d0d1"). InnerVolumeSpecName "kube-api-access-xq92d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:29.931586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.931560    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6" (OuterVolumeSpecName: "kube-api-access-kh2j6") pod "d352aaf3-620e-49b9-9be0-d960389b113c" (UID: "d352aaf3-620e-49b9-9be0-d960389b113c"). InnerVolumeSpecName "kube-api-access-kh2j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:29.942093 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.942066    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl2kw\" (UniqueName: \"kubernetes.io/projected/9a2f4b97-7fb7-4b92-ac37-7666a5f774f4-kube-api-access-kl2kw\") pod \"limitador-operator-controller-manager-74495c69f8-tsxwj\" (UID: \"9a2f4b97-7fb7-4b92-ac37-7666a5f774f4\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:29.957836 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:29.957774    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:30.026867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.026810    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d352aaf3-620e-49b9-9be0-d960389b113c-extensions-socket-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:30.026867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.026836    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xq92d\" (UniqueName: \"kubernetes.io/projected/f35d8a05-1b60-4287-8714-e4b0da93d0d1-kube-api-access-xq92d\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:30.026867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.026845    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh2j6\" (UniqueName: \"kubernetes.io/projected/d352aaf3-620e-49b9-9be0-d960389b113c-kube-api-access-kh2j6\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:30.112708 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.112670    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09823101-0324-4feb-94de-5a5bac71af98" path="/var/lib/kubelet/pods/09823101-0324-4feb-94de-5a5bac71af98/volumes"
Sep 10 16:24:30.112939 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.112928    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d352aaf3-620e-49b9-9be0-d960389b113c" path="/var/lib/kubelet/pods/d352aaf3-620e-49b9-9be0-d960389b113c/volumes"
Sep 10 16:24:30.113215 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.113203    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" path="/var/lib/kubelet/pods/f35d8a05-1b60-4287-8714-e4b0da93d0d1/volumes"
Sep 10 16:24:30.160063 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.160033    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:30.161655 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:30.161627    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11106ed1_759b_4873_b9a3_9ea6aa117828.slice/crio-8e19e48c4d99be6d36b9e71d1c664c3fb20c1d113fc5f838ee3e2efaa841270e WatchSource:0}: Error finding container 8e19e48c4d99be6d36b9e71d1c664c3fb20c1d113fc5f838ee3e2efaa841270e: Status 404 returned error can't find the container with id 8e19e48c4d99be6d36b9e71d1c664c3fb20c1d113fc5f838ee3e2efaa841270e
Sep 10 16:24:30.213383 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.213289    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:30.217512 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:30.217465    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab01006f_f96e_47de_bd3f_a787d47fbc2b.slice/crio-73084feaa9ecd86a7fd1147343a25ce7ee70b05af247154cad9a565c6934df61 WatchSource:0}: Error finding container 73084feaa9ecd86a7fd1147343a25ce7ee70b05af247154cad9a565c6934df61: Status 404 returned error can't find the container with id 73084feaa9ecd86a7fd1147343a25ce7ee70b05af247154cad9a565c6934df61
Sep 10 16:24:30.243874 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.243850    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"]
Sep 10 16:24:30.245267 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:30.245239    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a2f4b97_7fb7_4b92_ac37_7666a5f774f4.slice/crio-ccfa077eb0cc31727c93002ffd1b734482a55e51d94e2813ab14306540707f72 WatchSource:0}: Error finding container ccfa077eb0cc31727c93002ffd1b734482a55e51d94e2813ab14306540707f72: Status 404 returned error can't find the container with id ccfa077eb0cc31727c93002ffd1b734482a55e51d94e2813ab14306540707f72
Sep 10 16:24:30.347936 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.347897    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" event={"ID":"11106ed1-759b-4873-b9a3-9ea6aa117828","Type":"ContainerStarted","Data":"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"}
Sep 10 16:24:30.348097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.347960    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:30.348097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.347981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" event={"ID":"11106ed1-759b-4873-b9a3-9ea6aa117828","Type":"ContainerStarted","Data":"8e19e48c4d99be6d36b9e71d1c664c3fb20c1d113fc5f838ee3e2efaa841270e"}
Sep 10 16:24:30.349728 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.349704    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" event={"ID":"ab01006f-f96e-47de-bd3f-a787d47fbc2b","Type":"ContainerStarted","Data":"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"}
Sep 10 16:24:30.349827 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.349751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" event={"ID":"ab01006f-f96e-47de-bd3f-a787d47fbc2b","Type":"ContainerStarted","Data":"73084feaa9ecd86a7fd1147343a25ce7ee70b05af247154cad9a565c6934df61"}
Sep 10 16:24:30.349827 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.349791    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:30.350949 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.350929    2577 generic.go:356] "Generic (PLEG): container finished" podID="d352aaf3-620e-49b9-9be0-d960389b113c" containerID="2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03" exitCode=0
Sep 10 16:24:30.351045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.350961    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-bmlzk"
Sep 10 16:24:30.351045 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.351025    2577 scope.go:117] "RemoveContainer" containerID="2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03"
Sep 10 16:24:30.352482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.352458    2577 generic.go:356] "Generic (PLEG): container finished" podID="f35d8a05-1b60-4287-8714-e4b0da93d0d1" containerID="faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69" exitCode=0
Sep 10 16:24:30.352564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.352516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-dl8cw"
Sep 10 16:24:30.354483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.354464    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj" event={"ID":"9a2f4b97-7fb7-4b92-ac37-7666a5f774f4","Type":"ContainerStarted","Data":"05f5c7a328b6a3fcb3be7ffdd9e139341bf028d964eb1d9312799a52a08163dd"}
Sep 10 16:24:30.354614 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.354501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-mt6t4"
Sep 10 16:24:30.354804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.354504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj" event={"ID":"9a2f4b97-7fb7-4b92-ac37-7666a5f774f4","Type":"ContainerStarted","Data":"ccfa077eb0cc31727c93002ffd1b734482a55e51d94e2813ab14306540707f72"}
Sep 10 16:24:30.354804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.354750    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:30.361367 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.361346    2577 scope.go:117] "RemoveContainer" containerID="2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03"
Sep 10 16:24:30.361639 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:30.361621    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03\": container with ID starting with 2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03 not found: ID does not exist" containerID="2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03"
Sep 10 16:24:30.361694 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.361645    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03"} err="failed to get container status \"2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03\": rpc error: code = NotFound desc = could not find container \"2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03\": container with ID starting with 2ba26b39522a86405cc33401d6a2e36edfeb1f6c9637708a630a023e2ec37b03 not found: ID does not exist"
Sep 10 16:24:30.361694 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.361663    2577 scope.go:117] "RemoveContainer" containerID="faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69"
Sep 10 16:24:30.369138 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.369123    2577 scope.go:117] "RemoveContainer" containerID="faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69"
Sep 10 16:24:30.369428 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:30.369409    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69\": container with ID starting with faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69 not found: ID does not exist" containerID="faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69"
Sep 10 16:24:30.369496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.369437    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69"} err="failed to get container status \"faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69\": rpc error: code = NotFound desc = could not find container \"faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69\": container with ID starting with faa8cb3c75fe2cfb080a7144ca9e3cbba648b999e7c5cc5a003f7dd62d41de69 not found: ID does not exist"
Sep 10 16:24:30.377700 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.377645    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" podStartSLOduration=1.3776335020000001 podStartE2EDuration="1.377633502s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:30.375700317 +0000 UTC m=+442.777501645" watchObservedRunningTime="2026-09-10 16:24:30.377633502 +0000 UTC m=+442.779434629"
Sep 10 16:24:30.412636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.412584    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj" podStartSLOduration=1.412564054 podStartE2EDuration="1.412564054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:30.412208973 +0000 UTC m=+442.814010100" watchObservedRunningTime="2026-09-10 16:24:30.412564054 +0000 UTC m=+442.814365172"
Sep 10 16:24:30.446163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:30.446110    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" podStartSLOduration=1.446094799 podStartE2EDuration="1.446094799s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:30.445150741 +0000 UTC m=+442.846951880" watchObservedRunningTime="2026-09-10 16:24:30.446094799 +0000 UTC m=+442.847895926"
Sep 10 16:24:41.364802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.364767    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-tsxwj"
Sep 10 16:24:41.365379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.364839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:41.365379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.365169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:41.501264 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.501232    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:41.501633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.501568    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" podUID="11106ed1-759b-4873-b9a3-9ea6aa117828" containerName="manager" containerID="cri-o://b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5" gracePeriod=10
Sep 10 16:24:41.767831 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.767801    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:41.816760 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.816724    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:24:41.817106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.817094    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11106ed1-759b-4873-b9a3-9ea6aa117828" containerName="manager"
Sep 10 16:24:41.817153 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.817108    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="11106ed1-759b-4873-b9a3-9ea6aa117828" containerName="manager"
Sep 10 16:24:41.817188 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.817180    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="11106ed1-759b-4873-b9a3-9ea6aa117828" containerName="manager"
Sep 10 16:24:41.820306 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.820285    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume\") pod \"11106ed1-759b-4873-b9a3-9ea6aa117828\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") "
Sep 10 16:24:41.820402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.820346    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxbfn\" (UniqueName: \"kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn\") pod \"11106ed1-759b-4873-b9a3-9ea6aa117828\" (UID: \"11106ed1-759b-4873-b9a3-9ea6aa117828\") "
Sep 10 16:24:41.820594 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.820575    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:41.820673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.820621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "11106ed1-759b-4873-b9a3-9ea6aa117828" (UID: "11106ed1-759b-4873-b9a3-9ea6aa117828"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:24:41.822756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.822721    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn" (OuterVolumeSpecName: "kube-api-access-jxbfn") pod "11106ed1-759b-4873-b9a3-9ea6aa117828" (UID: "11106ed1-759b-4873-b9a3-9ea6aa117828"). InnerVolumeSpecName "kube-api-access-jxbfn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:41.845368 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.845337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:24:41.920908 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.920818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:41.920908 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.920867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crslp\" (UniqueName: \"kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:41.921097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.920973    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxbfn\" (UniqueName: \"kubernetes.io/projected/11106ed1-759b-4873-b9a3-9ea6aa117828-kube-api-access-jxbfn\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:41.921097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:41.920995    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/11106ed1-759b-4873-b9a3-9ea6aa117828-extensions-socket-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:42.021870 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.021829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crslp\" (UniqueName: \"kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:42.022060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.021918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:42.022262 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.022246    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:42.036047 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.036019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crslp\" (UniqueName: \"kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-fhlr5\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:42.137841 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.137804    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:42.425934 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.425900    2577 generic.go:356] "Generic (PLEG): container finished" podID="11106ed1-759b-4873-b9a3-9ea6aa117828" containerID="b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5" exitCode=0
Sep 10 16:24:42.426344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.425954    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"
Sep 10 16:24:42.426344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.425956    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" event={"ID":"11106ed1-759b-4873-b9a3-9ea6aa117828","Type":"ContainerDied","Data":"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"}
Sep 10 16:24:42.426344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.426081    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9" event={"ID":"11106ed1-759b-4873-b9a3-9ea6aa117828","Type":"ContainerDied","Data":"8e19e48c4d99be6d36b9e71d1c664c3fb20c1d113fc5f838ee3e2efaa841270e"}
Sep 10 16:24:42.426344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.426116    2577 scope.go:117] "RemoveContainer" containerID="b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"
Sep 10 16:24:42.426344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.426308    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:24:42.427287 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:24:42.427248    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88d5dc12_8352_415a_bc32_2590a7da7c7c.slice/crio-5db6889cf366d654f30337a2b6e2c22ca21f81895d4bb75109b1943526b08501 WatchSource:0}: Error finding container 5db6889cf366d654f30337a2b6e2c22ca21f81895d4bb75109b1943526b08501: Status 404 returned error can't find the container with id 5db6889cf366d654f30337a2b6e2c22ca21f81895d4bb75109b1943526b08501
Sep 10 16:24:42.434879 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.434860    2577 scope.go:117] "RemoveContainer" containerID="b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"
Sep 10 16:24:42.435162 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:42.435141    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5\": container with ID starting with b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5 not found: ID does not exist" containerID="b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"
Sep 10 16:24:42.435287 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.435173    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5"} err="failed to get container status \"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5\": rpc error: code = NotFound desc = could not find container \"b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5\": container with ID starting with b738f152f261eeb55867720c5672eb961ba1358e34d19200ce029516b085f5e5 not found: ID does not exist"
Sep 10 16:24:42.458120 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.458096    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:42.471963 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:42.471940    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-qwhv9"]
Sep 10 16:24:43.435064 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:43.435027    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" event={"ID":"88d5dc12-8352-415a-bc32-2590a7da7c7c","Type":"ContainerStarted","Data":"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"}
Sep 10 16:24:43.435575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:43.435092    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" event={"ID":"88d5dc12-8352-415a-bc32-2590a7da7c7c","Type":"ContainerStarted","Data":"5db6889cf366d654f30337a2b6e2c22ca21f81895d4bb75109b1943526b08501"}
Sep 10 16:24:43.435575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:43.435168    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:43.485885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:43.485827    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" podStartSLOduration=2.485806049 podStartE2EDuration="2.485806049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:24:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:24:43.484339316 +0000 UTC m=+455.886140449" watchObservedRunningTime="2026-09-10 16:24:43.485806049 +0000 UTC m=+455.887607178"
Sep 10 16:24:44.112165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:44.112130    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11106ed1-759b-4873-b9a3-9ea6aa117828" path="/var/lib/kubelet/pods/11106ed1-759b-4873-b9a3-9ea6aa117828/volumes"
Sep 10 16:24:54.443944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.443905    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:24:54.544216 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.544181    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:54.544442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.544413    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" podUID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" containerName="manager" containerID="cri-o://e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53" gracePeriod=10
Sep 10 16:24:54.800895 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.800871    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:54.810896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.810871    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume\") pod \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") "
Sep 10 16:24:54.811091 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.810909    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rbm9\" (UniqueName: \"kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9\") pod \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\" (UID: \"ab01006f-f96e-47de-bd3f-a787d47fbc2b\") "
Sep 10 16:24:54.811289 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.811252    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ab01006f-f96e-47de-bd3f-a787d47fbc2b" (UID: "ab01006f-f96e-47de-bd3f-a787d47fbc2b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:24:54.812934 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.812908    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9" (OuterVolumeSpecName: "kube-api-access-8rbm9") pod "ab01006f-f96e-47de-bd3f-a787d47fbc2b" (UID: "ab01006f-f96e-47de-bd3f-a787d47fbc2b"). InnerVolumeSpecName "kube-api-access-8rbm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:24:54.912128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.912090    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ab01006f-f96e-47de-bd3f-a787d47fbc2b-extensions-socket-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:54.912128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:54.912124    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rbm9\" (UniqueName: \"kubernetes.io/projected/ab01006f-f96e-47de-bd3f-a787d47fbc2b-kube-api-access-8rbm9\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:24:55.508801 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.508769    2577 generic.go:356] "Generic (PLEG): container finished" podID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" containerID="e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53" exitCode=0
Sep 10 16:24:55.509223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.508812    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"
Sep 10 16:24:55.509223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.508847    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" event={"ID":"ab01006f-f96e-47de-bd3f-a787d47fbc2b","Type":"ContainerDied","Data":"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"}
Sep 10 16:24:55.509223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.508888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94" event={"ID":"ab01006f-f96e-47de-bd3f-a787d47fbc2b","Type":"ContainerDied","Data":"73084feaa9ecd86a7fd1147343a25ce7ee70b05af247154cad9a565c6934df61"}
Sep 10 16:24:55.509223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.508908    2577 scope.go:117] "RemoveContainer" containerID="e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"
Sep 10 16:24:55.518519 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.518498    2577 scope.go:117] "RemoveContainer" containerID="e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"
Sep 10 16:24:55.518799 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:24:55.518778    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53\": container with ID starting with e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53 not found: ID does not exist" containerID="e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"
Sep 10 16:24:55.518865 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.518813    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53"} err="failed to get container status \"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53\": rpc error: code = NotFound desc = could not find container \"e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53\": container with ID starting with e0db958e81b0ee113353840a6aebc62ba64ee1604b644e45ed7197ab6a6ddf53 not found: ID does not exist"
Sep 10 16:24:55.538315 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.538266    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:55.547922 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:55.547888    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-89x94"]
Sep 10 16:24:56.112961 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:24:56.112926    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" path="/var/lib/kubelet/pods/ab01006f-f96e-47de-bd3f-a787d47fbc2b/volumes"
Sep 10 16:25:12.559252 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.559211    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:12.559702 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.559566    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" containerName="manager"
Sep 10 16:25:12.559702 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.559578    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" containerName="manager"
Sep 10 16:25:12.559702 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.559647    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab01006f-f96e-47de-bd3f-a787d47fbc2b" containerName="manager"
Sep 10 16:25:12.564060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.564043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.567033 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.567014    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 16:25:12.572819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.572799    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:12.640632 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.640596    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.640812 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.640787    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7fk\" (UniqueName: \"kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.647327 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.647296    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:12.742051 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.742013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7fk\" (UniqueName: \"kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.742233 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.742067    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.742691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.742671    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.765090 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.765061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7fk\" (UniqueName: \"kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk\") pod \"limitador-limitador-8588749977-r92qq\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") " pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:12.874807 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:12.874755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:13.155247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.155223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:13.158221 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:13.158193    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode173ada6_49d3_435c_882d_838f93b1eb0f.slice/crio-babce8400c95e16b39338150483a67659a89854b7194559de439e1f775cfeebb WatchSource:0}: Error finding container babce8400c95e16b39338150483a67659a89854b7194559de439e1f775cfeebb: Status 404 returned error can't find the container with id babce8400c95e16b39338150483a67659a89854b7194559de439e1f775cfeebb
Sep 10 16:25:13.385640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.385605    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:13.390403 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.390379    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:13.393478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.393458    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mrgst\""
Sep 10 16:25:13.399566 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.399543    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:13.448052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.447959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxtk8\" (UniqueName: \"kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8\") pod \"authorino-7bcfd8f9cd-sql8w\" (UID: \"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:13.549508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.549470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxtk8\" (UniqueName: \"kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8\") pod \"authorino-7bcfd8f9cd-sql8w\" (UID: \"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:13.559817 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.559776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxtk8\" (UniqueName: \"kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8\") pod \"authorino-7bcfd8f9cd-sql8w\" (UID: \"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893\") " pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:13.612040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.612007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" event={"ID":"e173ada6-49d3-435c-882d-838f93b1eb0f","Type":"ContainerStarted","Data":"babce8400c95e16b39338150483a67659a89854b7194559de439e1f775cfeebb"}
Sep 10 16:25:13.700105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.700019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:13.974422 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:13.974399    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:13.976413 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:13.976381    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9e7d4b5_1ec4_400d_b0d1_d35fc7eef893.slice/crio-c751925c128d2158a2bf726974da9affb60abae8c06d187f5a29dc7ac1a021ab WatchSource:0}: Error finding container c751925c128d2158a2bf726974da9affb60abae8c06d187f5a29dc7ac1a021ab: Status 404 returned error can't find the container with id c751925c128d2158a2bf726974da9affb60abae8c06d187f5a29dc7ac1a021ab
Sep 10 16:25:14.619699 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:14.619658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" event={"ID":"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893","Type":"ContainerStarted","Data":"c751925c128d2158a2bf726974da9affb60abae8c06d187f5a29dc7ac1a021ab"}
Sep 10 16:25:14.621154 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:14.621126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" event={"ID":"e173ada6-49d3-435c-882d-838f93b1eb0f","Type":"ContainerStarted","Data":"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"}
Sep 10 16:25:14.621321 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:14.621245    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:14.641692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:14.641637    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" podStartSLOduration=1.31756906 podStartE2EDuration="2.641620142s" totalImagesPullingTime="1.324051082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:14.640787352 +0000 UTC m=+487.042588479" watchObservedRunningTime="2026-09-10 16:25:14.641620142 +0000 UTC m=+487.043421331"
Sep 10 16:25:16.633170 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:16.633139    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" event={"ID":"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893","Type":"ContainerStarted","Data":"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"}
Sep 10 16:25:16.652368 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:16.652314    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" podStartSLOduration=1.6582291150000001 podStartE2EDuration="3.652298051s" totalImagesPullingTime="1.994068936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:16.649557789 +0000 UTC m=+489.051358910" watchObservedRunningTime="2026-09-10 16:25:16.652298051 +0000 UTC m=+489.054099169"
Sep 10 16:25:17.205977 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:17.205936    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:18.644478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:18.644412    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" podUID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" containerName="authorino" containerID="cri-o://2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c" gracePeriod=30
Sep 10 16:25:18.895612 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:18.895559    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:19.000751 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.000715    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fxtk8\" (UniqueName: \"kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8\") pod \"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893\" (UID: \"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893\") "
Sep 10 16:25:19.002845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.002810    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8" (OuterVolumeSpecName: "kube-api-access-fxtk8") pod "f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" (UID: "f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893"). InnerVolumeSpecName "kube-api-access-fxtk8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:19.101356 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.101319    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fxtk8\" (UniqueName: \"kubernetes.io/projected/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893-kube-api-access-fxtk8\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:25:19.650844 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.650810    2577 generic.go:356] "Generic (PLEG): container finished" podID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" containerID="2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c" exitCode=0
Sep 10 16:25:19.651316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.650851    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w"
Sep 10 16:25:19.651316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.650883    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" event={"ID":"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893","Type":"ContainerDied","Data":"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"}
Sep 10 16:25:19.651316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.650922    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-sql8w" event={"ID":"f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893","Type":"ContainerDied","Data":"c751925c128d2158a2bf726974da9affb60abae8c06d187f5a29dc7ac1a021ab"}
Sep 10 16:25:19.651316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.650939    2577 scope.go:117] "RemoveContainer" containerID="2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"
Sep 10 16:25:19.660024 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.660004    2577 scope.go:117] "RemoveContainer" containerID="2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"
Sep 10 16:25:19.660299 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:19.660264    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c\": container with ID starting with 2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c not found: ID does not exist" containerID="2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"
Sep 10 16:25:19.660377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.660307    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c"} err="failed to get container status \"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c\": rpc error: code = NotFound desc = could not find container \"2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c\": container with ID starting with 2ed8d88514a34e0988584f7f1fd18e725686930347be7b73f0068cf9f1506c8c not found: ID does not exist"
Sep 10 16:25:19.677109 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.677076    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:19.679198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:19.679178    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-sql8w"]
Sep 10 16:25:20.112559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:20.112537    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" path="/var/lib/kubelet/pods/f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893/volumes"
Sep 10 16:25:25.628161 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:25.628127    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:27.058838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.058801    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:27.059218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.059019    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" podUID="e173ada6-49d3-435c-882d-838f93b1eb0f" containerName="limitador" containerID="cri-o://9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82" gracePeriod=30
Sep 10 16:25:27.613445 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.613416    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:27.679864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.679771    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z7fk\" (UniqueName: \"kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk\") pod \"e173ada6-49d3-435c-882d-838f93b1eb0f\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") "
Sep 10 16:25:27.679864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.679815    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file\") pod \"e173ada6-49d3-435c-882d-838f93b1eb0f\" (UID: \"e173ada6-49d3-435c-882d-838f93b1eb0f\") "
Sep 10 16:25:27.680226 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.680202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file" (OuterVolumeSpecName: "config-file") pod "e173ada6-49d3-435c-882d-838f93b1eb0f" (UID: "e173ada6-49d3-435c-882d-838f93b1eb0f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:25:27.681879 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.681848    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk" (OuterVolumeSpecName: "kube-api-access-6z7fk") pod "e173ada6-49d3-435c-882d-838f93b1eb0f" (UID: "e173ada6-49d3-435c-882d-838f93b1eb0f"). InnerVolumeSpecName "kube-api-access-6z7fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:27.700366 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.700322    2577 generic.go:356] "Generic (PLEG): container finished" podID="e173ada6-49d3-435c-882d-838f93b1eb0f" containerID="9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82" exitCode=0
Sep 10 16:25:27.700535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.700372    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-r92qq"
Sep 10 16:25:27.700535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.700402    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" event={"ID":"e173ada6-49d3-435c-882d-838f93b1eb0f","Type":"ContainerDied","Data":"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"}
Sep 10 16:25:27.700535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.700454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-r92qq" event={"ID":"e173ada6-49d3-435c-882d-838f93b1eb0f","Type":"ContainerDied","Data":"babce8400c95e16b39338150483a67659a89854b7194559de439e1f775cfeebb"}
Sep 10 16:25:27.700535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.700477    2577 scope.go:117] "RemoveContainer" containerID="9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"
Sep 10 16:25:27.709541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.709521    2577 scope.go:117] "RemoveContainer" containerID="9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"
Sep 10 16:25:27.709841 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:27.709815    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82\": container with ID starting with 9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82 not found: ID does not exist" containerID="9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"
Sep 10 16:25:27.709895 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.709851    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82"} err="failed to get container status \"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82\": rpc error: code = NotFound desc = could not find container \"9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82\": container with ID starting with 9142fea0e84c6b4eab221de04f3c6a9cf346e732d86a8541948204cf0cc7ea82 not found: ID does not exist"
Sep 10 16:25:27.724636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.724604    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:27.730516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.730489    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-r92qq"]
Sep 10 16:25:27.780831 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.780793    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z7fk\" (UniqueName: \"kubernetes.io/projected/e173ada6-49d3-435c-882d-838f93b1eb0f-kube-api-access-6z7fk\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:25:27.780831 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:27.780824    2577 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/e173ada6-49d3-435c-882d-838f93b1eb0f-config-file\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:25:28.112011 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:28.111933    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e173ada6-49d3-435c-882d-838f93b1eb0f" path="/var/lib/kubelet/pods/e173ada6-49d3-435c-882d-838f93b1eb0f/volumes"
Sep 10 16:25:31.930139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930105    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"]
Sep 10 16:25:31.930561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930546    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" containerName="authorino"
Sep 10 16:25:31.930616 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930562    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" containerName="authorino"
Sep 10 16:25:31.930616 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930588    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e173ada6-49d3-435c-882d-838f93b1eb0f" containerName="limitador"
Sep 10 16:25:31.930616 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930594    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e173ada6-49d3-435c-882d-838f93b1eb0f" containerName="limitador"
Sep 10 16:25:31.930710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930648    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e173ada6-49d3-435c-882d-838f93b1eb0f" containerName="limitador"
Sep 10 16:25:31.930710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.930663    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e7d4b5-1ec4-400d-b0d1-d35fc7eef893" containerName="authorino"
Sep 10 16:25:31.933672 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.933652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:31.937923 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.937898    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 16:25:31.939353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.939330    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-9g4lh\""
Sep 10 16:25:31.939453 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.939404    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 16:25:31.939602 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.939589    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 16:25:31.947303 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:31.947256    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"]
Sep 10 16:25:32.018131 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.018097    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4486v\" (UniqueName: \"kubernetes.io/projected/538982de-b548-4af5-a889-27ad60adf5cf-kube-api-access-4486v\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.018303 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.018141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/538982de-b548-4af5-a889-27ad60adf5cf-data\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.118695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.118664    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4486v\" (UniqueName: \"kubernetes.io/projected/538982de-b548-4af5-a889-27ad60adf5cf-kube-api-access-4486v\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.118864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.118709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/538982de-b548-4af5-a889-27ad60adf5cf-data\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.119081 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.119063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/538982de-b548-4af5-a889-27ad60adf5cf-data\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.129400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.129368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4486v\" (UniqueName: \"kubernetes.io/projected/538982de-b548-4af5-a889-27ad60adf5cf-kube-api-access-4486v\") pod \"postgres-69fb4594cf-hhvqk\" (UID: \"538982de-b548-4af5-a889-27ad60adf5cf\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.244333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.244220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:32.520497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.520465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"]
Sep 10 16:25:32.521871 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:32.521843    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod538982de_b548_4af5_a889_27ad60adf5cf.slice/crio-ae0e42729464425a3fe6bd211491a37f85780b4e18e435b68701fd2b89709bb5 WatchSource:0}: Error finding container ae0e42729464425a3fe6bd211491a37f85780b4e18e435b68701fd2b89709bb5: Status 404 returned error can't find the container with id ae0e42729464425a3fe6bd211491a37f85780b4e18e435b68701fd2b89709bb5
Sep 10 16:25:32.729641 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:32.729607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk" event={"ID":"538982de-b548-4af5-a889-27ad60adf5cf","Type":"ContainerStarted","Data":"ae0e42729464425a3fe6bd211491a37f85780b4e18e435b68701fd2b89709bb5"}
Sep 10 16:25:35.748837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:35.748801    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk" event={"ID":"538982de-b548-4af5-a889-27ad60adf5cf","Type":"ContainerStarted","Data":"0a5392e12357ddb85aa3298944300d3f1f55752cd4a58d9f0e2782ba0ce041e8"}
Sep 10 16:25:35.749268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:35.748859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:35.770269 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:35.770215    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk" podStartSLOduration=2.579764518 podStartE2EDuration="4.770201619s" totalImagesPullingTime="2.190437101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:35.767235091 +0000 UTC m=+508.169036217" watchObservedRunningTime="2026-09-10 16:25:35.770201619 +0000 UTC m=+508.172002785"
Sep 10 16:25:41.784500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:41.784463    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-hhvqk"
Sep 10 16:25:42.373815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.373772    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-nj7tt"]
Sep 10 16:25:42.380509 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.380485    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.383507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.383482    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-mrgst\""
Sep 10 16:25:42.392465 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.392446    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-nj7tt"]
Sep 10 16:25:42.502592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.502561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhvnl\" (UniqueName: \"kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl\") pod \"authorino-557f876f69-nj7tt\" (UID: \"f37a2a55-8cac-45e7-a2ba-869ebecbad55\") " pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.603426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.603390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhvnl\" (UniqueName: \"kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl\") pod \"authorino-557f876f69-nj7tt\" (UID: \"f37a2a55-8cac-45e7-a2ba-869ebecbad55\") " pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.610431 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.610399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nj7tt"]
Sep 10 16:25:42.610717 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:42.610696    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rhvnl], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-nj7tt" podUID="f37a2a55-8cac-45e7-a2ba-869ebecbad55"
Sep 10 16:25:42.618408 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.618382    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhvnl\" (UniqueName: \"kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl\") pod \"authorino-557f876f69-nj7tt\" (UID: \"f37a2a55-8cac-45e7-a2ba-869ebecbad55\") " pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.646162 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.646082    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:42.650648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.650630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:42.660109 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.660084    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:42.794298 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.790110    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.802416 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.802392    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:42.805417 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.805395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkn2b\" (UniqueName: \"kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b\") pod \"authorino-6dfc959bd6-x86wh\" (UID: \"0b998480-b6e5-42e4-9785-39c5660a6dc1\") " pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:42.906376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.906271    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhvnl\" (UniqueName: \"kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl\") pod \"f37a2a55-8cac-45e7-a2ba-869ebecbad55\" (UID: \"f37a2a55-8cac-45e7-a2ba-869ebecbad55\") "
Sep 10 16:25:42.906527 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.906507    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkn2b\" (UniqueName: \"kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b\") pod \"authorino-6dfc959bd6-x86wh\" (UID: \"0b998480-b6e5-42e4-9785-39c5660a6dc1\") " pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:42.908455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.908430    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl" (OuterVolumeSpecName: "kube-api-access-rhvnl") pod "f37a2a55-8cac-45e7-a2ba-869ebecbad55" (UID: "f37a2a55-8cac-45e7-a2ba-869ebecbad55"). InnerVolumeSpecName "kube-api-access-rhvnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:42.916479 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.916455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkn2b\" (UniqueName: \"kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b\") pod \"authorino-6dfc959bd6-x86wh\" (UID: \"0b998480-b6e5-42e4-9785-39c5660a6dc1\") " pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:42.961244 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:42.961211    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:43.007300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.007249    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhvnl\" (UniqueName: \"kubernetes.io/projected/f37a2a55-8cac-45e7-a2ba-869ebecbad55-kube-api-access-rhvnl\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:25:43.244620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.244591    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:43.246427 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:43.246399    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b998480_b6e5_42e4_9785_39c5660a6dc1.slice/crio-314919815cebaf039f73ce916d0525c79961fc6391e22de9a180f8e8386c898d WatchSource:0}: Error finding container 314919815cebaf039f73ce916d0525c79961fc6391e22de9a180f8e8386c898d: Status 404 returned error can't find the container with id 314919815cebaf039f73ce916d0525c79961fc6391e22de9a180f8e8386c898d
Sep 10 16:25:43.796806 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.796766    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" event={"ID":"0b998480-b6e5-42e4-9785-39c5660a6dc1","Type":"ContainerStarted","Data":"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"}
Sep 10 16:25:43.796806 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.796809    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" event={"ID":"0b998480-b6e5-42e4-9785-39c5660a6dc1","Type":"ContainerStarted","Data":"314919815cebaf039f73ce916d0525c79961fc6391e22de9a180f8e8386c898d"}
Sep 10 16:25:43.797215 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.796830    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-nj7tt"
Sep 10 16:25:43.851195 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.851089    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" podStartSLOduration=1.5158616999999999 podStartE2EDuration="1.851071032s" totalImagesPullingTime="335.209332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:43.826485131 +0000 UTC m=+516.228286259" watchObservedRunningTime="2026-09-10 16:25:43.851071032 +0000 UTC m=+516.252872177"
Sep 10 16:25:43.867526 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.867495    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-nj7tt"]
Sep 10 16:25:43.874655 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:43.874627    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-nj7tt"]
Sep 10 16:25:44.004896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.004856    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:25:44.008439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.008418    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.011884 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.011864    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 16:25:44.024460 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.024432    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:25:44.112781 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.112754    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f37a2a55-8cac-45e7-a2ba-869ebecbad55" path="/var/lib/kubelet/pods/f37a2a55-8cac-45e7-a2ba-869ebecbad55/volumes"
Sep 10 16:25:44.118100 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.118064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79v6j\" (UniqueName: \"kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.118249 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.118115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.218630 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.218596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.218805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.218688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79v6j\" (UniqueName: \"kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.221185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.221160    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.230211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.230177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79v6j\" (UniqueName: \"kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j\") pod \"authorino-66447b7744-tm92w\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") " pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.316977 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.316937    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:25:44.594234 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.594206    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:25:44.595891 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:44.595866    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3ea6cd0_46d9_4025_ae7a_0afa145794fa.slice/crio-3461e35abc6a5c1474885a3bdea6eb9a3801e455896afe51027aeed5a929a713 WatchSource:0}: Error finding container 3461e35abc6a5c1474885a3bdea6eb9a3801e455896afe51027aeed5a929a713: Status 404 returned error can't find the container with id 3461e35abc6a5c1474885a3bdea6eb9a3801e455896afe51027aeed5a929a713
Sep 10 16:25:44.803575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:44.803490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66447b7744-tm92w" event={"ID":"c3ea6cd0-46d9-4025-ae7a-0afa145794fa","Type":"ContainerStarted","Data":"3461e35abc6a5c1474885a3bdea6eb9a3801e455896afe51027aeed5a929a713"}
Sep 10 16:25:45.810651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:45.810615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66447b7744-tm92w" event={"ID":"c3ea6cd0-46d9-4025-ae7a-0afa145794fa","Type":"ContainerStarted","Data":"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"}
Sep 10 16:25:45.834864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:45.834806    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66447b7744-tm92w" podStartSLOduration=2.383803195 podStartE2EDuration="2.834789463s" totalImagesPullingTime="450.986268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:45.833803154 +0000 UTC m=+518.235604282" watchObservedRunningTime="2026-09-10 16:25:45.834789463 +0000 UTC m=+518.236590589"
Sep 10 16:25:45.869459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:45.869425    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:45.869648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:45.869596    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" podUID="0b998480-b6e5-42e4-9785-39c5660a6dc1" containerName="authorino" containerID="cri-o://da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9" gracePeriod=30
Sep 10 16:25:46.133346 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.133317    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:46.235913 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.235878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lkn2b\" (UniqueName: \"kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b\") pod \"0b998480-b6e5-42e4-9785-39c5660a6dc1\" (UID: \"0b998480-b6e5-42e4-9785-39c5660a6dc1\") "
Sep 10 16:25:46.238020 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.237996    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b" (OuterVolumeSpecName: "kube-api-access-lkn2b") pod "0b998480-b6e5-42e4-9785-39c5660a6dc1" (UID: "0b998480-b6e5-42e4-9785-39c5660a6dc1"). InnerVolumeSpecName "kube-api-access-lkn2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:25:46.337414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.337380    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lkn2b\" (UniqueName: \"kubernetes.io/projected/0b998480-b6e5-42e4-9785-39c5660a6dc1-kube-api-access-lkn2b\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:25:46.816999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.816964    2577 generic.go:356] "Generic (PLEG): container finished" podID="0b998480-b6e5-42e4-9785-39c5660a6dc1" containerID="da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9" exitCode=0
Sep 10 16:25:46.817426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.817003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dfc959bd6-x86wh"
Sep 10 16:25:46.817426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.817047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" event={"ID":"0b998480-b6e5-42e4-9785-39c5660a6dc1","Type":"ContainerDied","Data":"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"}
Sep 10 16:25:46.817426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.817083    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dfc959bd6-x86wh" event={"ID":"0b998480-b6e5-42e4-9785-39c5660a6dc1","Type":"ContainerDied","Data":"314919815cebaf039f73ce916d0525c79961fc6391e22de9a180f8e8386c898d"}
Sep 10 16:25:46.817426 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.817099    2577 scope.go:117] "RemoveContainer" containerID="da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"
Sep 10 16:25:46.829300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.827675    2577 scope.go:117] "RemoveContainer" containerID="da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"
Sep 10 16:25:46.829300 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:46.828058    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9\": container with ID starting with da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9 not found: ID does not exist" containerID="da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"
Sep 10 16:25:46.829300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.828099    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9"} err="failed to get container status \"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9\": rpc error: code = NotFound desc = could not find container \"da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9\": container with ID starting with da8c4f640ac288dc5669e5bc835c943549eb862b8ed98a8d51d4d2ea4a40b0a9 not found: ID does not exist"
Sep 10 16:25:46.848078 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.848051    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:46.854167 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:46.854132    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dfc959bd6-x86wh"]
Sep 10 16:25:48.112398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.112365    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b998480-b6e5-42e4-9785-39c5660a6dc1" path="/var/lib/kubelet/pods/0b998480-b6e5-42e4-9785-39c5660a6dc1/volumes"
Sep 10 16:25:48.662398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.662360    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:25:48.662799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.662780    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b998480-b6e5-42e4-9785-39c5660a6dc1" containerName="authorino"
Sep 10 16:25:48.662887 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.662802    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b998480-b6e5-42e4-9785-39c5660a6dc1" containerName="authorino"
Sep 10 16:25:48.662945 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.662925    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b998480-b6e5-42e4-9785-39c5660a6dc1" containerName="authorino"
Sep 10 16:25:48.667516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.667494    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.670672 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.670651    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:25:48.670672 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.670658    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:25:48.670875 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.670753    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:25:48.670875 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.670770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tl6vz\""
Sep 10 16:25:48.676864 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.676844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:25:48.758496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.758460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.758663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.758518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5fzh\" (UniqueName: \"kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.758663 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.758553    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.851453 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.851421    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:25:48.855180 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.855157    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:48.859288 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.859246    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5fzh\" (UniqueName: \"kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.859409 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.859313    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.859409 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.859401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.859544 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:48.859511    2577 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 16:25:48.859598 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:25:48.859572    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs podName:59c0dd42-4a4d-412f-8829-7a6c7bcd770f nodeName:}" failed. No retries permitted until 2026-09-10 16:25:49.359553206 +0000 UTC m=+521.761354317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs") pod "maas-controller-56769c848c-2n7qk" (UID: "59c0dd42-4a4d-412f-8829-7a6c7bcd770f") : secret "maas-controller-metrics-tls" not found
Sep 10 16:25:48.862592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.862323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.866786 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.866760    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:25:48.882800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.882771    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5fzh\" (UniqueName: \"kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:48.960297 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.960179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:48.960297 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.960229    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:48.960484 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:48.960364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sg67d\" (UniqueName: \"kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.061687 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.061650    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sg67d\" (UniqueName: \"kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.061869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.061701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.061869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.061742    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.064350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.064323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.064481 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.064417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.071756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.071733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sg67d\" (UniqueName: \"kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d\") pod \"maas-controller-7bb45c94d6-hzntg\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") " pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.175349 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.175266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:49.365121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.365086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:49.367512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.367484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") pod \"maas-controller-56769c848c-2n7qk\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") " pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:49.459671 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.459637    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:25:49.461304 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:49.461253    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3f10bd_8e11_4ffc_8392_d8bb3cc84844.slice/crio-4f39de27f611d45f29674d0cdc442fdd7519976ae2e89b010ff08207a505db9c WatchSource:0}: Error finding container 4f39de27f611d45f29674d0cdc442fdd7519976ae2e89b010ff08207a505db9c: Status 404 returned error can't find the container with id 4f39de27f611d45f29674d0cdc442fdd7519976ae2e89b010ff08207a505db9c
Sep 10 16:25:49.580996 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.580903    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:49.837645 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.837571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" event={"ID":"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844","Type":"ContainerStarted","Data":"4f39de27f611d45f29674d0cdc442fdd7519976ae2e89b010ff08207a505db9c"}
Sep 10 16:25:49.860972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:49.860948    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:25:49.862625 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:25:49.862599    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c0dd42_4a4d_412f_8829_7a6c7bcd770f.slice/crio-27e2e6c049a2c9336d34bb86711d36957bde70e3ec0a1b30e22c6b421f5c3fdf WatchSource:0}: Error finding container 27e2e6c049a2c9336d34bb86711d36957bde70e3ec0a1b30e22c6b421f5c3fdf: Status 404 returned error can't find the container with id 27e2e6c049a2c9336d34bb86711d36957bde70e3ec0a1b30e22c6b421f5c3fdf
Sep 10 16:25:50.847734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:50.847647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56769c848c-2n7qk" event={"ID":"59c0dd42-4a4d-412f-8829-7a6c7bcd770f","Type":"ContainerStarted","Data":"27e2e6c049a2c9336d34bb86711d36957bde70e3ec0a1b30e22c6b421f5c3fdf"}
Sep 10 16:25:52.863186 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.863136    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56769c848c-2n7qk" event={"ID":"59c0dd42-4a4d-412f-8829-7a6c7bcd770f","Type":"ContainerStarted","Data":"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"}
Sep 10 16:25:52.863850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.863216    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:25:52.864889 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.864859    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" event={"ID":"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844","Type":"ContainerStarted","Data":"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"}
Sep 10 16:25:52.864984 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.864908    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:25:52.890749 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.890694    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56769c848c-2n7qk" podStartSLOduration=2.758481369 podStartE2EDuration="4.890676019s" totalImagesPullingTime="2.13219465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:52.88761174 +0000 UTC m=+525.289412866" watchObservedRunningTime="2026-09-10 16:25:52.890676019 +0000 UTC m=+525.292477145"
Sep 10 16:25:52.915995 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:25:52.915939    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" podStartSLOduration=2.393700624 podStartE2EDuration="4.915920409s" totalImagesPullingTime="2.522219785s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:25:52.915366667 +0000 UTC m=+525.317167795" watchObservedRunningTime="2026-09-10 16:25:52.915920409 +0000 UTC m=+525.317721536"
Sep 10 16:26:03.873624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:03.873547    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:26:03.873624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:03.873619    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:26:03.940400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:03.940362    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:26:03.940753 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:03.940637    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56769c848c-2n7qk" podUID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" containerName="manager" containerID="cri-o://8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6" gracePeriod=30
Sep 10 16:26:04.191350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.191320    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:26:04.196836 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.196813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert\") pod \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") "
Sep 10 16:26:04.196949 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.196855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5fzh\" (UniqueName: \"kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh\") pod \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") "
Sep 10 16:26:04.196949 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.196906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") pod \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\" (UID: \"59c0dd42-4a4d-412f-8829-7a6c7bcd770f\") "
Sep 10 16:26:04.199016 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.198986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "59c0dd42-4a4d-412f-8829-7a6c7bcd770f" (UID: "59c0dd42-4a4d-412f-8829-7a6c7bcd770f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:26:04.199121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.199017    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh" (OuterVolumeSpecName: "kube-api-access-t5fzh") pod "59c0dd42-4a4d-412f-8829-7a6c7bcd770f" (UID: "59c0dd42-4a4d-412f-8829-7a6c7bcd770f"). InnerVolumeSpecName "kube-api-access-t5fzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:26:04.199121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.199046    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "59c0dd42-4a4d-412f-8829-7a6c7bcd770f" (UID: "59c0dd42-4a4d-412f-8829-7a6c7bcd770f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:26:04.298068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.298027    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-webhook-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:26:04.298068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.298062    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5fzh\" (UniqueName: \"kubernetes.io/projected/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-kube-api-access-t5fzh\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:26:04.298068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.298072    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59c0dd42-4a4d-412f-8829-7a6c7bcd770f-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:26:04.932986 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.932950    2577 generic.go:356] "Generic (PLEG): container finished" podID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" containerID="8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6" exitCode=0
Sep 10 16:26:04.933391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.932997    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56769c848c-2n7qk"
Sep 10 16:26:04.933391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.933041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56769c848c-2n7qk" event={"ID":"59c0dd42-4a4d-412f-8829-7a6c7bcd770f","Type":"ContainerDied","Data":"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"}
Sep 10 16:26:04.933391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.933091    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56769c848c-2n7qk" event={"ID":"59c0dd42-4a4d-412f-8829-7a6c7bcd770f","Type":"ContainerDied","Data":"27e2e6c049a2c9336d34bb86711d36957bde70e3ec0a1b30e22c6b421f5c3fdf"}
Sep 10 16:26:04.933391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.933113    2577 scope.go:117] "RemoveContainer" containerID="8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"
Sep 10 16:26:04.943171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.942975    2577 scope.go:117] "RemoveContainer" containerID="8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"
Sep 10 16:26:04.943305 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:26:04.943269    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6\": container with ID starting with 8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6 not found: ID does not exist" containerID="8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"
Sep 10 16:26:04.943351 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.943314    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6"} err="failed to get container status \"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6\": rpc error: code = NotFound desc = could not find container \"8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6\": container with ID starting with 8d4c5a0eada5c760798d9a2c2fa93f79657f397e1e74a3140f7e4dacc32ea1e6 not found: ID does not exist"
Sep 10 16:26:04.956644 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.956622    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:26:04.961446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:04.961426    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56769c848c-2n7qk"]
Sep 10 16:26:06.112788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:06.112755    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" path="/var/lib/kubelet/pods/59c0dd42-4a4d-412f-8829-7a6c7bcd770f/volumes"
Sep 10 16:26:26.939432 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.939392    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:26:26.939965 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.939948    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" containerName="manager"
Sep 10 16:26:26.940012 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.939969    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" containerName="manager"
Sep 10 16:26:26.940102 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.940091    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c0dd42-4a4d-412f-8829-7a6c7bcd770f" containerName="manager"
Sep 10 16:26:26.946526 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.946502    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:26.950674 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.950655    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 16:26:26.952107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.952086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5q8xx\""
Sep 10 16:26:26.952202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.952086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 16:26:26.961515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.961492    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:26:26.979212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.979174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhh8c\" (UniqueName: \"kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:26.979378 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.979224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:26.979378 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.979253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:26.979378 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:26.979348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.080207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.080173    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhh8c\" (UniqueName: \"kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.080207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.080215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.080498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.080236    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.080498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.080258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.080498 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:26:27.080371    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 16:26:27.080498 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:26:27.080425    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs podName:319286b9-4ff5-4560-98bf-6fc2d8630862 nodeName:}" failed. No retries permitted until 2026-09-10 16:26:27.580406512 +0000 UTC m=+559.982207617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs") pod "maas-api-6c68568df5-f8x2t" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862") : secret "maas-api-metrics-tls" not found
Sep 10 16:26:27.081127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.081100    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.082646 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.082620    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.092856 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.092825    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhh8c\" (UniqueName: \"kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.564680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.564646    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"]
Sep 10 16:26:27.568309 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.568288    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.573416 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.573387    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 16:26:27.573707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.573687    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-95vl7\""
Sep 10 16:26:27.585343 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.585314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:27.585728 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:26:27.585618    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 16:26:27.585911 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:26:27.585901    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs podName:319286b9-4ff5-4560-98bf-6fc2d8630862 nodeName:}" failed. No retries permitted until 2026-09-10 16:26:28.585879831 +0000 UTC m=+560.987680941 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs") pod "maas-api-6c68568df5-f8x2t" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862") : secret "maas-api-metrics-tls" not found
Sep 10 16:26:27.588755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.588734    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"]
Sep 10 16:26:27.686964 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.686932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1a84f0-74ab-417c-ad76-a66f38857ee5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.687149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.686975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sm7gp\" (UniqueName: \"kubernetes.io/projected/8a1a84f0-74ab-417c-ad76-a66f38857ee5-kube-api-access-sm7gp\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.788482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.788432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1a84f0-74ab-417c-ad76-a66f38857ee5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.788482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.788487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sm7gp\" (UniqueName: \"kubernetes.io/projected/8a1a84f0-74ab-417c-ad76-a66f38857ee5-kube-api-access-sm7gp\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.789460 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.789426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8a1a84f0-74ab-417c-ad76-a66f38857ee5-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.799007 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.798980    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sm7gp\" (UniqueName: \"kubernetes.io/projected/8a1a84f0-74ab-417c-ad76-a66f38857ee5-kube-api-access-sm7gp\") pod \"payload-pre-processing-59fbc47bf8-k2fkh\" (UID: \"8a1a84f0-74ab-417c-ad76-a66f38857ee5\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:27.879892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:27.879853    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:28.168185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:28.168159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"]
Sep 10 16:26:28.169098 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:26:28.169074    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a1a84f0_74ab_417c_ad76_a66f38857ee5.slice/crio-725f712af71c90004f54412565d04cacf1a9735bf24b79e7a227f3c84be40f6b WatchSource:0}: Error finding container 725f712af71c90004f54412565d04cacf1a9735bf24b79e7a227f3c84be40f6b: Status 404 returned error can't find the container with id 725f712af71c90004f54412565d04cacf1a9735bf24b79e7a227f3c84be40f6b
Sep 10 16:26:28.596347 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:28.596210    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:28.598835 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:28.598811    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"maas-api-6c68568df5-f8x2t\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") " pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:28.758128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:28.758083    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:29.042848 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:29.042824    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:26:29.045167 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:26:29.045126    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod319286b9_4ff5_4560_98bf_6fc2d8630862.slice/crio-e60ee81e765b48024c9e5959c77347535ca1e71eeae03c089d5648a64ad8e717 WatchSource:0}: Error finding container e60ee81e765b48024c9e5959c77347535ca1e71eeae03c089d5648a64ad8e717: Status 404 returned error can't find the container with id e60ee81e765b48024c9e5959c77347535ca1e71eeae03c089d5648a64ad8e717
Sep 10 16:26:29.076536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:29.076492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" event={"ID":"319286b9-4ff5-4560-98bf-6fc2d8630862","Type":"ContainerStarted","Data":"e60ee81e765b48024c9e5959c77347535ca1e71eeae03c089d5648a64ad8e717"}
Sep 10 16:26:29.078043 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:29.078016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh" event={"ID":"8a1a84f0-74ab-417c-ad76-a66f38857ee5","Type":"ContainerStarted","Data":"725f712af71c90004f54412565d04cacf1a9735bf24b79e7a227f3c84be40f6b"}
Sep 10 16:26:30.085822 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:30.085780    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh" event={"ID":"8a1a84f0-74ab-417c-ad76-a66f38857ee5","Type":"ContainerStarted","Data":"1689c792362b56861bb590218a1339c37cdfb5aa10f37defd34a01ad3af31844"}
Sep 10 16:26:30.086224 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:30.085986    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:30.107408 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:30.107354    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh" podStartSLOduration=1.364183033 podStartE2EDuration="3.107334266s" totalImagesPullingTime="1.743151233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:26:30.105869014 +0000 UTC m=+562.507670143" watchObservedRunningTime="2026-09-10 16:26:30.107334266 +0000 UTC m=+562.509135392"
Sep 10 16:26:32.100567 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:32.100467    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" event={"ID":"319286b9-4ff5-4560-98bf-6fc2d8630862","Type":"ContainerStarted","Data":"7ad7884c9508c35da99017e99cc309805f274446832ac096ddec11ca69f11335"}
Sep 10 16:26:32.100932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:32.100572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:32.130648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:32.130598    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" podStartSLOduration=3.495910583 podStartE2EDuration="6.130582404s" totalImagesPullingTime="2.634671821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:26:32.129345909 +0000 UTC m=+564.531147039" watchObservedRunningTime="2026-09-10 16:26:32.130582404 +0000 UTC m=+564.532383549"
Sep 10 16:26:38.113813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:38.113785    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:26:41.095778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:41.095742    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-k2fkh"
Sep 10 16:26:55.242266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.242233    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"]
Sep 10 16:26:55.247057 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.247036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.251309 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.251270    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 16:26:55.251421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.251270    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 16:26:55.251421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.251286    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 16:26:55.251582 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.251448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h86xf\""
Sep 10 16:26:55.257839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.257810    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"]
Sep 10 16:26:55.322754 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.322999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322766    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.322999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322786    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f745e6b-7bfc-4055-8f04-0cb94023e435-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.322999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.322999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322919    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.322999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.322954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jqrv\" (UniqueName: \"kubernetes.io/projected/1f745e6b-7bfc-4055-8f04-0cb94023e435-kube-api-access-4jqrv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424454 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424417    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424487    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f745e6b-7bfc-4055-8f04-0cb94023e435-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424546    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424627 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jqrv\" (UniqueName: \"kubernetes.io/projected/1f745e6b-7bfc-4055-8f04-0cb94023e435-kube-api-access-4jqrv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.424926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.425012 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424913    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.425012 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.424951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.427127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.427095    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1f745e6b-7bfc-4055-8f04-0cb94023e435-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.427226 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.427108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1f745e6b-7bfc-4055-8f04-0cb94023e435-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.433205 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.433183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jqrv\" (UniqueName: \"kubernetes.io/projected/1f745e6b-7bfc-4055-8f04-0cb94023e435-kube-api-access-4jqrv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rzj5d\" (UID: \"1f745e6b-7bfc-4055-8f04-0cb94023e435\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.560766 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.560678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:26:55.839379 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:55.839354    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"]
Sep 10 16:26:55.841184 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:26:55.841157    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f745e6b_7bfc_4055_8f04_0cb94023e435.slice/crio-b68b9f11f8d63cb92dc9095db6ee64d53bbf3c5876fcccb2992466e900d0321f WatchSource:0}: Error finding container b68b9f11f8d63cb92dc9095db6ee64d53bbf3c5876fcccb2992466e900d0321f: Status 404 returned error can't find the container with id b68b9f11f8d63cb92dc9095db6ee64d53bbf3c5876fcccb2992466e900d0321f
Sep 10 16:26:56.248519 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:56.248481    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d" event={"ID":"1f745e6b-7bfc-4055-8f04-0cb94023e435","Type":"ContainerStarted","Data":"b68b9f11f8d63cb92dc9095db6ee64d53bbf3c5876fcccb2992466e900d0321f"}
Sep 10 16:26:57.256337 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:57.256220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d" event={"ID":"1f745e6b-7bfc-4055-8f04-0cb94023e435","Type":"ContainerStarted","Data":"34736a259b45f4d70398213e5353f9ebf6ad294672e509ea459e471ad5f3d9e7"}
Sep 10 16:26:57.277611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:26:57.277561    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d" podStartSLOduration=1.193502727 podStartE2EDuration="2.27754715s" totalImagesPullingTime="1.084044423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:26:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:26:57.274499419 +0000 UTC m=+589.676300546" watchObservedRunningTime="2026-09-10 16:26:57.27754715 +0000 UTC m=+589.679348277"
Sep 10 16:27:05.561204 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:05.561160    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:27:05.561631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:05.561221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:27:05.573704 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:05.573678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:27:06.323938 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:06.323902    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rzj5d"
Sep 10 16:27:08.064407 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:08.064381    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:27:08.065204 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:08.065186    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:27:24.645082 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.645040    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"]
Sep 10 16:27:24.649211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.649190    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.652134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.652108    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 16:27:24.657656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.657627    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"]
Sep 10 16:27:24.677370 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.677541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.677541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.677541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677534    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.677711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6015082-18e4-4b0a-b776-5d53164e8b17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.677711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.677698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpc4x\" (UniqueName: \"kubernetes.io/projected/d6015082-18e4-4b0a-b776-5d53164e8b17-kube-api-access-dpc4x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.778764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778718    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6015082-18e4-4b0a-b776-5d53164e8b17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.778764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778771    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpc4x\" (UniqueName: \"kubernetes.io/projected/d6015082-18e4-4b0a-b776-5d53164e8b17-kube-api-access-dpc4x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778796    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.778879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.779262    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779411 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.779330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.779451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.779411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.781193 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.781165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6015082-18e4-4b0a-b776-5d53164e8b17-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.781312 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.781260    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6015082-18e4-4b0a-b776-5d53164e8b17-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.789492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.789465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpc4x\" (UniqueName: \"kubernetes.io/projected/d6015082-18e4-4b0a-b776-5d53164e8b17-kube-api-access-dpc4x\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9\" (UID: \"d6015082-18e4-4b0a-b776-5d53164e8b17\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:24.964185 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:24.964104    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:25.241350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:25.241327    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"]
Sep 10 16:27:25.242996 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:27:25.242966    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6015082_18e4_4b0a_b776_5d53164e8b17.slice/crio-ce417422884a098d0cb029b78e45f7043a1dff74c5458d65138d8ddc4eb66444 WatchSource:0}: Error finding container ce417422884a098d0cb029b78e45f7043a1dff74c5458d65138d8ddc4eb66444: Status 404 returned error can't find the container with id ce417422884a098d0cb029b78e45f7043a1dff74c5458d65138d8ddc4eb66444
Sep 10 16:27:25.244332 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:25.244312    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:27:25.425221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:25.425187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9" event={"ID":"d6015082-18e4-4b0a-b776-5d53164e8b17","Type":"ContainerStarted","Data":"ce417422884a098d0cb029b78e45f7043a1dff74c5458d65138d8ddc4eb66444"}
Sep 10 16:27:26.433738 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:26.433698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9" event={"ID":"d6015082-18e4-4b0a-b776-5d53164e8b17","Type":"ContainerStarted","Data":"c9646f59fb0e4d77fa7cbcc0a66fa01904ba68577abd6992e14b260663c3dc96"}
Sep 10 16:27:26.455620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:26.455560    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9" podStartSLOduration=2.214164759 podStartE2EDuration="2.455545472s" totalImagesPullingTime="241.380713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:27:26.453815606 +0000 UTC m=+618.855616734" watchObservedRunningTime="2026-09-10 16:27:26.455545472 +0000 UTC m=+618.857346599"
Sep 10 16:27:34.964523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:34.964437    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:34.964970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:34.964553    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:34.977705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:34.977674    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:27:35.498557 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:27:35.498527    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9"
Sep 10 16:29:04.091812 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.091718    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:29:04.092328 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.091933    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-66447b7744-tm92w" podUID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" containerName="authorino" containerID="cri-o://3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684" gracePeriod=30
Sep 10 16:29:04.335815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.335780    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:29:04.372132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.372095    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert\") pod \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") "
Sep 10 16:29:04.372310 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.372167    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79v6j\" (UniqueName: \"kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j\") pod \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\" (UID: \"c3ea6cd0-46d9-4025-ae7a-0afa145794fa\") "
Sep 10 16:29:04.374169 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.374140    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j" (OuterVolumeSpecName: "kube-api-access-79v6j") pod "c3ea6cd0-46d9-4025-ae7a-0afa145794fa" (UID: "c3ea6cd0-46d9-4025-ae7a-0afa145794fa"). InnerVolumeSpecName "kube-api-access-79v6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:29:04.385442 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.385405    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c3ea6cd0-46d9-4025-ae7a-0afa145794fa" (UID: "c3ea6cd0-46d9-4025-ae7a-0afa145794fa"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:29:04.473310 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.473237    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-tls-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:29:04.473310 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:04.473302    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79v6j\" (UniqueName: \"kubernetes.io/projected/c3ea6cd0-46d9-4025-ae7a-0afa145794fa-kube-api-access-79v6j\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:29:05.016589 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.016549    2577 generic.go:356] "Generic (PLEG): container finished" podID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" containerID="3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684" exitCode=0
Sep 10 16:29:05.016792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.016599    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66447b7744-tm92w"
Sep 10 16:29:05.016792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.016638    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66447b7744-tm92w" event={"ID":"c3ea6cd0-46d9-4025-ae7a-0afa145794fa","Type":"ContainerDied","Data":"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"}
Sep 10 16:29:05.016792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.016687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66447b7744-tm92w" event={"ID":"c3ea6cd0-46d9-4025-ae7a-0afa145794fa","Type":"ContainerDied","Data":"3461e35abc6a5c1474885a3bdea6eb9a3801e455896afe51027aeed5a929a713"}
Sep 10 16:29:05.016792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.016709    2577 scope.go:117] "RemoveContainer" containerID="3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"
Sep 10 16:29:05.026789 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.026774    2577 scope.go:117] "RemoveContainer" containerID="3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"
Sep 10 16:29:05.027053 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:05.027038    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684\": container with ID starting with 3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684 not found: ID does not exist" containerID="3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"
Sep 10 16:29:05.027097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.027060    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684"} err="failed to get container status \"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684\": rpc error: code = NotFound desc = could not find container \"3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684\": container with ID starting with 3f2b287e468ca9e6f6fcefe9cb45a9eb5860f630b32495af01b52151d53f4684 not found: ID does not exist"
Sep 10 16:29:05.046096 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.046065    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:29:05.048380 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:05.048360    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66447b7744-tm92w"]
Sep 10 16:29:06.113430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:06.113395    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" path="/var/lib/kubelet/pods/c3ea6cd0-46d9-4025-ae7a-0afa145794fa/volumes"
Sep 10 16:29:08.132073 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:08.132035    2577 scope.go:117] "RemoveContainer" containerID="847a0a88d7352dc09f540807bfbd1926ea16cf3cabafc5c6ed86310a4e0cf298"
Sep 10 16:29:21.159062 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.158992    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:29:21.159618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.159474    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" containerName="authorino"
Sep 10 16:29:21.159618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.159489    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" containerName="authorino"
Sep 10 16:29:21.159618 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.159575    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3ea6cd0-46d9-4025-ae7a-0afa145794fa" containerName="authorino"
Sep 10 16:29:21.164242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.164224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.167536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.167513    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1abdaf-openshift-default-dockercfg-k6jq7\""
Sep 10 16:29:21.168179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.168161    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 16:29:21.181364 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.181328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:29:21.225930 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.225892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.225947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.225976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226225 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226225 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226190    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226357 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k897l\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.226518 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.226493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.327783 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.327783 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327784    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327843    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.327994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328041    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k897l\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328236    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328764    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.328815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.329030 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.328904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.330540 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.330519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.330851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.330834    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.338072 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.338038    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.338348 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.338330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k897l\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l\") pod \"e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") " pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.474678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.474592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:21.757691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:21.757665    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:29:21.762983 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:21.762863    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ce9f22d_f82a_44ab_b72c_b83ace64023d.slice/crio-d9caaff33495db9e07fb5cad27a2640339fc875d324da8d782b1f325fde5fda4 WatchSource:0}: Error finding container d9caaff33495db9e07fb5cad27a2640339fc875d324da8d782b1f325fde5fda4: Status 404 returned error can't find the container with id d9caaff33495db9e07fb5cad27a2640339fc875d324da8d782b1f325fde5fda4
Sep 10 16:29:22.118057 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.118021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerStarted","Data":"d9caaff33495db9e07fb5cad27a2640339fc875d324da8d782b1f325fde5fda4"}
Sep 10 16:29:22.758746 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.758709    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7688-5b04c8"]
Sep 10 16:29:22.762653 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.762629    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:22.765978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.765951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mt6ps\""
Sep 10 16:29:22.775774 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.775750    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-5b04c8"]
Sep 10 16:29:22.841374 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.841338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2vqc\" (UniqueName: \"kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc\") pod \"test-curl-7688-5b04c8\" (UID: \"3b1fc92a-af4c-47c1-bf76-57d1caa2352f\") " pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:22.942651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.942605    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2vqc\" (UniqueName: \"kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc\") pod \"test-curl-7688-5b04c8\" (UID: \"3b1fc92a-af4c-47c1-bf76-57d1caa2352f\") " pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:22.952534 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:22.952507    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2vqc\" (UniqueName: \"kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc\") pod \"test-curl-7688-5b04c8\" (UID: \"3b1fc92a-af4c-47c1-bf76-57d1caa2352f\") " pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:23.074482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:23.074391    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:23.378457 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:23.378429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-5b04c8"]
Sep 10 16:29:23.381340 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:23.381308    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3b1fc92a_af4c_47c1_bf76_57d1caa2352f.slice/crio-b6472634e191586866b5b68f400996802f425724b846f79e2d7ebe7e4dff72c7 WatchSource:0}: Error finding container b6472634e191586866b5b68f400996802f425724b846f79e2d7ebe7e4dff72c7: Status 404 returned error can't find the container with id b6472634e191586866b5b68f400996802f425724b846f79e2d7ebe7e4dff72c7
Sep 10 16:29:24.133198 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:24.133153    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-5b04c8" event={"ID":"3b1fc92a-af4c-47c1-bf76-57d1caa2352f","Type":"ContainerStarted","Data":"b6472634e191586866b5b68f400996802f425724b846f79e2d7ebe7e4dff72c7"}
Sep 10 16:29:24.497126 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:24.497085    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:24.497242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:24.497158    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:24.497242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:24.497188    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:25.142497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:25.142453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerStarted","Data":"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"}
Sep 10 16:29:25.173490 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:25.173436    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podStartSLOduration=1.441388437 podStartE2EDuration="4.173412862s" totalImagesPullingTime="2.732024425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:25.172595231 +0000 UTC m=+737.574396360" watchObservedRunningTime="2026-09-10 16:29:25.173412862 +0000 UTC m=+737.575213991"
Sep 10 16:29:25.475797 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:25.475713    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:25.481802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:25.481770    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:26.149785 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.149747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-5b04c8" event={"ID":"3b1fc92a-af4c-47c1-bf76-57d1caa2352f","Type":"ContainerStarted","Data":"09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f"}
Sep 10 16:29:26.150153 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.150137    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:26.151094 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.151075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:26.168525 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.168479    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7688-5b04c8" podStartSLOduration=1.592007342 podStartE2EDuration="4.168464314s" totalImagesPullingTime="2.576456972s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:26.166676511 +0000 UTC m=+738.568477637" watchObservedRunningTime="2026-09-10 16:29:26.168464314 +0000 UTC m=+738.570265441"
Sep 10 16:29:26.754470 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.754427    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7688-5b04c8"]
Sep 10 16:29:26.760331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.760302    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7688-5b04c8"]
Sep 10 16:29:26.882106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.882071    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7688-6ea861"]
Sep 10 16:29:26.882658 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.882632    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" containerName="test-curl-7688-5b04c8"
Sep 10 16:29:26.882658 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.882658    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" containerName="test-curl-7688-5b04c8"
Sep 10 16:29:26.882844 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.882753    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" containerName="test-curl-7688-5b04c8"
Sep 10 16:29:26.886516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.886495    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:26.896056 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.896000    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-6ea861"]
Sep 10 16:29:26.986099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:26.986056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2jl5\" (UniqueName: \"kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5\") pod \"test-curl-7688-6ea861\" (UID: \"291a2741-c5c1-4485-81a7-dc28ab250ea0\") " pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:27.087675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:27.087587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2jl5\" (UniqueName: \"kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5\") pod \"test-curl-7688-6ea861\" (UID: \"291a2741-c5c1-4485-81a7-dc28ab250ea0\") " pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:27.105131 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:27.105096    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2jl5\" (UniqueName: \"kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5\") pod \"test-curl-7688-6ea861\" (UID: \"291a2741-c5c1-4485-81a7-dc28ab250ea0\") " pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:27.156069 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:27.156034    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7688-5b04c8" container="test-curl-7688-5b04c8"
Sep 10 16:29:27.199207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:27.199179    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:27.486002 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:27.485967    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7688-6ea861"]
Sep 10 16:29:27.486200 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:27.486174    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod291a2741_c5c1_4485_81a7_dc28ab250ea0.slice/crio-26397b91d2f3e879add928bbac3b2fe27cca9bd33fc58cfed2a776d722d6c1f8 WatchSource:0}: Error finding container 26397b91d2f3e879add928bbac3b2fe27cca9bd33fc58cfed2a776d722d6c1f8: Status 404 returned error can't find the container with id 26397b91d2f3e879add928bbac3b2fe27cca9bd33fc58cfed2a776d722d6c1f8
Sep 10 16:29:28.162677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.162643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-6ea861" event={"ID":"291a2741-c5c1-4485-81a7-dc28ab250ea0","Type":"ContainerStarted","Data":"3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9"}
Sep 10 16:29:28.162677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.162692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7688-6ea861" event={"ID":"291a2741-c5c1-4485-81a7-dc28ab250ea0","Type":"ContainerStarted","Data":"26397b91d2f3e879add928bbac3b2fe27cca9bd33fc58cfed2a776d722d6c1f8"}
Sep 10 16:29:28.163111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.163000    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7688-5b04c8" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" containerName="test-curl-7688-5b04c8" containerID="cri-o://09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f" gracePeriod=2
Sep 10 16:29:28.185237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.185181    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7688-6ea861" podStartSLOduration=2.185159827 podStartE2EDuration="2.185159827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:28.179668001 +0000 UTC m=+740.581469128" watchObservedRunningTime="2026-09-10 16:29:28.185159827 +0000 UTC m=+740.586960955"
Sep 10 16:29:28.366761 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.366726    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:29:28.381017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.380985    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.382024 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.382001    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:29:28.385690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.385103    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-1abdaf-serving-cert\""
Sep 10 16:29:28.500038 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.499932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.500038 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.499986    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjrmq\" (UniqueName: \"kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.500303 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.500127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.500303 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.500210    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.601098 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.601064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.601232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.601118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.601325 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.601306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.601392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.601346    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjrmq\" (UniqueName: \"kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.601464 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:28.601447    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-serving-cert: secret "maas-api-e2e-inf-a-1abdaf-serving-cert" not found
Sep 10 16:29:28.601540 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:28.601528    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls podName:aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f nodeName:}" failed. No retries permitted until 2026-09-10 16:29:29.101508141 +0000 UTC m=+741.503309251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls") pod "maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" (UID: "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f") : secret "maas-api-e2e-inf-a-1abdaf-serving-cert" not found
Sep 10 16:29:28.601926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.601903    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.603458 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.603440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.611463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.611440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjrmq\" (UniqueName: \"kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:28.830089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.830009    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7688-6ea861"]
Sep 10 16:29:28.836515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:28.836489    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7688-6ea861"]
Sep 10 16:29:29.106520 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:29.106424    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:29.108816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:29.108783    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") pod \"maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:29.168161 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:29.168128    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7688-6ea861" container="test-curl-7688-6ea861"
Sep 10 16:29:29.298095 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:29.298060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:29.578156 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:29.578113    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:29:29.578928 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:29.578902    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9b9bcb_5f6e_4aa8_b38b_427f6911d85f.slice/crio-f3e4686eedb97c377e3d328f810d32ef0355eca03625556ef13619d031cc478b WatchSource:0}: Error finding container f3e4686eedb97c377e3d328f810d32ef0355eca03625556ef13619d031cc478b: Status 404 returned error can't find the container with id f3e4686eedb97c377e3d328f810d32ef0355eca03625556ef13619d031cc478b
Sep 10 16:29:30.174989 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.174960    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" event={"ID":"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f","Type":"ContainerStarted","Data":"f3e4686eedb97c377e3d328f810d32ef0355eca03625556ef13619d031cc478b"}
Sep 10 16:29:30.175347 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.175100    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7688-6ea861" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" containerName="test-curl-7688-6ea861" containerID="cri-o://3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9" gracePeriod=2
Sep 10 16:29:30.362834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.362810    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:30.520556 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.520469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2vqc\" (UniqueName: \"kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc\") pod \"3b1fc92a-af4c-47c1-bf76-57d1caa2352f\" (UID: \"3b1fc92a-af4c-47c1-bf76-57d1caa2352f\") "
Sep 10 16:29:30.522463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.522439    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc" (OuterVolumeSpecName: "kube-api-access-t2vqc") pod "3b1fc92a-af4c-47c1-bf76-57d1caa2352f" (UID: "3b1fc92a-af4c-47c1-bf76-57d1caa2352f"). InnerVolumeSpecName "kube-api-access-t2vqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:29:30.621611 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:30.621576    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2vqc\" (UniqueName: \"kubernetes.io/projected/3b1fc92a-af4c-47c1-bf76-57d1caa2352f-kube-api-access-t2vqc\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:29:31.183302 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.183243    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" event={"ID":"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f","Type":"ContainerStarted","Data":"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"}
Sep 10 16:29:31.184352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.183540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:31.185475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.185443    2577 generic.go:356] "Generic (PLEG): container finished" podID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" containerID="09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f" exitCode=137
Sep 10 16:29:31.185592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.185529    2577 scope.go:117] "RemoveContainer" containerID="09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f"
Sep 10 16:29:31.185697 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.185682    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:31.196107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.196085    2577 scope.go:117] "RemoveContainer" containerID="09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f"
Sep 10 16:29:31.196423 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:31.196400    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f\": container with ID starting with 09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f not found: ID does not exist" containerID="09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f"
Sep 10 16:29:31.196482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.196432    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f"} err="failed to get container status \"09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f\": rpc error: code = NotFound desc = could not find container \"09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f\": container with ID starting with 09f45bebdbc1107d1b603e67ab3c36e216be761e0879e42c3242a1c93285459f not found: ID does not exist"
Sep 10 16:29:31.205346 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:31.205310    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7688-5b04c8\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" pod="opendatahub/test-curl-7688-5b04c8"
Sep 10 16:29:31.205490 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:31.205450    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" podStartSLOduration=2.579286441 podStartE2EDuration="3.205435294s" totalImagesPullingTime="626.148853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:31.202675895 +0000 UTC m=+743.604477021" watchObservedRunningTime="2026-09-10 16:29:31.205435294 +0000 UTC m=+743.607236420"
Sep 10 16:29:32.114190 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:32.114149    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b1fc92a-af4c-47c1-bf76-57d1caa2352f" path="/var/lib/kubelet/pods/3b1fc92a-af4c-47c1-bf76-57d1caa2352f/volumes"
Sep 10 16:29:32.369119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:32.369057    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:32.447208 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:32.447178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r2jl5\" (UniqueName: \"kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5\") pod \"291a2741-c5c1-4485-81a7-dc28ab250ea0\" (UID: \"291a2741-c5c1-4485-81a7-dc28ab250ea0\") "
Sep 10 16:29:32.449318 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:32.449264    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5" (OuterVolumeSpecName: "kube-api-access-r2jl5") pod "291a2741-c5c1-4485-81a7-dc28ab250ea0" (UID: "291a2741-c5c1-4485-81a7-dc28ab250ea0"). InnerVolumeSpecName "kube-api-access-r2jl5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:29:32.548004 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:32.547969    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r2jl5\" (UniqueName: \"kubernetes.io/projected/291a2741-c5c1-4485-81a7-dc28ab250ea0-kube-api-access-r2jl5\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:29:33.202488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:33.202452    2577 generic.go:356] "Generic (PLEG): container finished" podID="291a2741-c5c1-4485-81a7-dc28ab250ea0" containerID="3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9" exitCode=137
Sep 10 16:29:33.202643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:33.202497    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:33.202643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:33.202555    2577 scope.go:117] "RemoveContainer" containerID="3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9"
Sep 10 16:29:33.212191 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:33.212172    2577 scope.go:117] "RemoveContainer" containerID="3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9"
Sep 10 16:29:33.212539 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:33.212509    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9\": container with ID starting with 3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9 not found: ID does not exist" containerID="3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9"
Sep 10 16:29:33.212639 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:33.212550    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9"} err="failed to get container status \"3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9\": rpc error: code = NotFound desc = could not find container \"3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9\": container with ID starting with 3d134777533f72fbe9225f8c419de741349907151029c06c809677a8e5d9dfd9 not found: ID does not exist"
Sep 10 16:29:33.214751 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:33.214726    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7688-6ea861\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" pod="opendatahub/test-curl-7688-6ea861"
Sep 10 16:29:34.114639 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.114608    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" path="/var/lib/kubelet/pods/291a2741-c5c1-4485-81a7-dc28ab250ea0/volumes"
Sep 10 16:29:34.877085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.877053    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:29:34.877648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.877627    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" containerName="test-curl-7688-6ea861"
Sep 10 16:29:34.877648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.877648    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" containerName="test-curl-7688-6ea861"
Sep 10 16:29:34.877824 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.877718    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="291a2741-c5c1-4485-81a7-dc28ab250ea0" containerName="test-curl-7688-6ea861"
Sep 10 16:29:34.882191 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.882170    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:34.885579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.885554    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-2d818c-serving-cert\""
Sep 10 16:29:34.897026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.897002    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:29:34.966352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.966323    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dzm6\" (UniqueName: \"kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:34.966495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.966375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:34.966495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.966444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:34.966581 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:34.966518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.067322 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.067286    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.067488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.067349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.067488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.067386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.067488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.067449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dzm6\" (UniqueName: \"kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.067607 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:35.067559    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-serving-cert: secret "maas-api-e2e-autores-2d818c-serving-cert" not found
Sep 10 16:29:35.067647 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:29:35.067628    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls podName:e83d586b-db35-4f7b-b419-b865e0d5cf96 nodeName:}" failed. No retries permitted until 2026-09-10 16:29:35.567607523 +0000 UTC m=+747.969408647 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls") pod "maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" (UID: "e83d586b-db35-4f7b-b419-b865e0d5cf96") : secret "maas-api-e2e-autores-2d818c-serving-cert" not found
Sep 10 16:29:35.068052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.068027    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.069931 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.069909    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.076550 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.076526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dzm6\" (UniqueName: \"kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.571051 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.571011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.574139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.574112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") pod \"maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.771384 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.771341    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:29:35.777123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.777096    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.782076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.781641    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-2d818c-dockercfg-qqxj8\""
Sep 10 16:29:35.782076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.781793    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-2d818c\""
Sep 10 16:29:35.784128 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.784100    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:29:35.795194 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.795167    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:35.874004 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.873949    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glxxm\" (UniqueName: \"kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.874125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.874007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.974700 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.974662    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glxxm\" (UniqueName: \"kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.974700 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.974703    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.975477 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.975456    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:35.986388 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:35.985069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glxxm\" (UniqueName: \"kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm\") pod \"payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:36.079090 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:36.079063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:29:36.080583 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:36.080559    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode83d586b_db35_4f7b_b419_b865e0d5cf96.slice/crio-c48b502aabe2880e6453fd45e02385387b39f5deffdc064cd13138e58d54d765 WatchSource:0}: Error finding container c48b502aabe2880e6453fd45e02385387b39f5deffdc064cd13138e58d54d765: Status 404 returned error can't find the container with id c48b502aabe2880e6453fd45e02385387b39f5deffdc064cd13138e58d54d765
Sep 10 16:29:36.095740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:36.095714    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:36.229174 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:36.229132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" event={"ID":"e83d586b-db35-4f7b-b419-b865e0d5cf96","Type":"ContainerStarted","Data":"c48b502aabe2880e6453fd45e02385387b39f5deffdc064cd13138e58d54d765"}
Sep 10 16:29:36.382114 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:36.382083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:29:36.383925 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:36.383883    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ed2ae9_01a4_4388_af0b_b5d7288c03c0.slice/crio-08e8d00472d41816beaad7fd7705215d79ac471e1c446ba344841f438187fd39 WatchSource:0}: Error finding container 08e8d00472d41816beaad7fd7705215d79ac471e1c446ba344841f438187fd39: Status 404 returned error can't find the container with id 08e8d00472d41816beaad7fd7705215d79ac471e1c446ba344841f438187fd39
Sep 10 16:29:37.209575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.209540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:29:37.237636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.237597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" event={"ID":"48ed2ae9-01a4-4388-af0b-b5d7288c03c0","Type":"ContainerStarted","Data":"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"}
Sep 10 16:29:37.237805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.237660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" event={"ID":"48ed2ae9-01a4-4388-af0b-b5d7288c03c0","Type":"ContainerStarted","Data":"08e8d00472d41816beaad7fd7705215d79ac471e1c446ba344841f438187fd39"}
Sep 10 16:29:37.237805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.237791    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:37.239177 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.239147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" event={"ID":"e83d586b-db35-4f7b-b419-b865e0d5cf96","Type":"ContainerStarted","Data":"8b9251e814c8f68068186b5c0a8e7d30aeb441e705ab1407d1439246f3c26725"}
Sep 10 16:29:37.239342 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.239191    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:37.263603 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.263548    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" podStartSLOduration=2.263527955 podStartE2EDuration="2.263527955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:37.257847961 +0000 UTC m=+749.659649089" watchObservedRunningTime="2026-09-10 16:29:37.263527955 +0000 UTC m=+749.665329082"
Sep 10 16:29:37.280144 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:37.280094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" podStartSLOduration=2.910162698 podStartE2EDuration="3.280077305s" totalImagesPullingTime="369.914607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:37.278484486 +0000 UTC m=+749.680285614" watchObservedRunningTime="2026-09-10 16:29:37.280077305 +0000 UTC m=+749.681878431"
Sep 10 16:29:38.111308 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.111256    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:29:38.135295 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.134982    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:29:38.135471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.135077    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.138202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.138029    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-25d77f0b-serving-cert\""
Sep 10 16:29:38.319619 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.319575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.320074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.319630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbqzv\" (UniqueName: \"kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.320074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.319731    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.320074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.319754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.420467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.420427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.420640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.420516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbqzv\" (UniqueName: \"kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.420640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.420606    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.420640 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.420629    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.421092 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.421039    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.423551 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.423525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.423639 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.423571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.432628 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.432599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbqzv\" (UniqueName: \"kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv\") pod \"maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.451449 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.451421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:38.762990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:38.762717    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:29:38.764701 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:38.764675    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e06e127_0bba_40ba_abc4_5393d62d4a0f.slice/crio-367e339ff609a24622e8cf054ec59623f288fa9a3543e9ea24194e6a5151abc8 WatchSource:0}: Error finding container 367e339ff609a24622e8cf054ec59623f288fa9a3543e9ea24194e6a5151abc8: Status 404 returned error can't find the container with id 367e339ff609a24622e8cf054ec59623f288fa9a3543e9ea24194e6a5151abc8
Sep 10 16:29:39.253975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:39.253940    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" event={"ID":"6e06e127-0bba-40ba-abc4-5393d62d4a0f","Type":"ContainerStarted","Data":"367e339ff609a24622e8cf054ec59623f288fa9a3543e9ea24194e6a5151abc8"}
Sep 10 16:29:40.261401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:40.261363    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" event={"ID":"6e06e127-0bba-40ba-abc4-5393d62d4a0f","Type":"ContainerStarted","Data":"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"}
Sep 10 16:29:40.261774 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:40.261487    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:40.299300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:40.298985    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" podStartSLOduration=1.774232745 podStartE2EDuration="2.29896038s" totalImagesPullingTime="524.727635ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:40.297608884 +0000 UTC m=+752.699410008" watchObservedRunningTime="2026-09-10 16:29:40.29896038 +0000 UTC m=+752.700761509"
Sep 10 16:29:43.252511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:43.252477    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:29:46.274048 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.273970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:29:46.314769 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.314563    2577 generic.go:356] "Generic (PLEG): container finished" podID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerID="3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153" exitCode=0
Sep 10 16:29:46.314769 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.314643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerDied","Data":"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"}
Sep 10 16:29:46.315796 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.315767    2577 scope.go:117] "RemoveContainer" containerID="3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"
Sep 10 16:29:46.317591 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.317190    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:46.317591 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.317311    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:46.317591 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:46.317377    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:29:47.322343 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:47.322302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerStarted","Data":"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"}
Sep 10 16:29:47.475677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:47.475637    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:47.475882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:47.475858    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:47.475933 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:47.475912    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:48.249240 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:48.249209    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:29:48.475701 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:48.475667    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:48.476089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:48.475725    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:49.246390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.245834    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:29:49.252423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.252399    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.257061 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.257001    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-9f6082-serving-cert\""
Sep 10 16:29:49.311295 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.309413    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:29:49.418958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.418927    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.419266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.419057    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9pp\" (UniqueName: \"kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.419266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.419168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.419266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.419217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.476016 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.475968    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:49.476475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.476018    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:49.520643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.520544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.520643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.520589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9pp\" (UniqueName: \"kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.520643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.520624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.520643 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.520643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.521406 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.521380    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.523067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.523047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.523149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.523072    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.529905 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.529883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9pp\" (UniqueName: \"kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp\") pod \"maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.564844 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.564808    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:49.874120 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:49.874090    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:29:49.874924 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:29:49.874898    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41f77fd_d833_459a_aea9_69ed25c1f3cb.slice/crio-ba01a3140651ae3fb32c0946fbd9f52597eaa3c18d1c30e3dc0a06b09eff3e44 WatchSource:0}: Error finding container ba01a3140651ae3fb32c0946fbd9f52597eaa3c18d1c30e3dc0a06b09eff3e44: Status 404 returned error can't find the container with id ba01a3140651ae3fb32c0946fbd9f52597eaa3c18d1c30e3dc0a06b09eff3e44
Sep 10 16:29:50.353133 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.353094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" event={"ID":"c41f77fd-d833-459a-aea9-69ed25c1f3cb","Type":"ContainerStarted","Data":"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"}
Sep 10 16:29:50.353331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.353176    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" event={"ID":"c41f77fd-d833-459a-aea9-69ed25c1f3cb","Type":"ContainerStarted","Data":"ba01a3140651ae3fb32c0946fbd9f52597eaa3c18d1c30e3dc0a06b09eff3e44"}
Sep 10 16:29:50.353331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.353211    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:29:50.400414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.396399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" podStartSLOduration=1.058260966 podStartE2EDuration="1.396376144s" totalImagesPullingTime="338.115178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:29:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:29:50.395504948 +0000 UTC m=+762.797306074" watchObservedRunningTime="2026-09-10 16:29:50.396376144 +0000 UTC m=+762.798177266"
Sep 10 16:29:50.475951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.475869    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:50.476175 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:50.476140    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:51.475682 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:51.475647    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:51.475892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:51.475703    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:52.475451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:52.475410    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body=
Sep 10 16:29:52.475919 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:52.475481    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused"
Sep 10 16:29:53.478808 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:53.478778    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:53.479211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:53.479189    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:53.479839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:53.479806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:29:56.365950 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:29:56.365917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:30:15.844529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:15.844495    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:30:15.844891 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:15.844729    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" podUID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" containerName="maas-api" containerID="cri-o://fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449" gracePeriod=30
Sep 10 16:30:16.119137 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.119102    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:30:16.256621 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.256590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls\") pod \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") "
Sep 10 16:30:16.256928 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.256910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs\") pod \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") "
Sep 10 16:30:16.257088 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.257074    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbqzv\" (UniqueName: \"kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv\") pod \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") "
Sep 10 16:30:16.257209 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.257197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca\") pod \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\" (UID: \"6e06e127-0bba-40ba-abc4-5393d62d4a0f\") "
Sep 10 16:30:16.257655 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.257623    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e06e127-0bba-40ba-abc4-5393d62d4a0f" (UID: "6e06e127-0bba-40ba-abc4-5393d62d4a0f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:30:16.259491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.259466    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e06e127-0bba-40ba-abc4-5393d62d4a0f" (UID: "6e06e127-0bba-40ba-abc4-5393d62d4a0f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:16.260550 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.260517    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6e06e127-0bba-40ba-abc4-5393d62d4a0f" (UID: "6e06e127-0bba-40ba-abc4-5393d62d4a0f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:16.260656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.260626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv" (OuterVolumeSpecName: "kube-api-access-rbqzv") pod "6e06e127-0bba-40ba-abc4-5393d62d4a0f" (UID: "6e06e127-0bba-40ba-abc4-5393d62d4a0f"). InnerVolumeSpecName "kube-api-access-rbqzv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:30:16.358773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.358689    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:16.358773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.358718    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbqzv\" (UniqueName: \"kubernetes.io/projected/6e06e127-0bba-40ba-abc4-5393d62d4a0f-kube-api-access-rbqzv\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:16.358773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.358728    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e06e127-0bba-40ba-abc4-5393d62d4a0f-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:16.358773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.358738    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e06e127-0bba-40ba-abc4-5393d62d4a0f-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:16.512421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.512385    2577 generic.go:356] "Generic (PLEG): container finished" podID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" containerID="fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449" exitCode=0
Sep 10 16:30:16.512606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.512421    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" event={"ID":"6e06e127-0bba-40ba-abc4-5393d62d4a0f","Type":"ContainerDied","Data":"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"}
Sep 10 16:30:16.512606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.512442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"
Sep 10 16:30:16.512606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.512493    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv" event={"ID":"6e06e127-0bba-40ba-abc4-5393d62d4a0f","Type":"ContainerDied","Data":"367e339ff609a24622e8cf054ec59623f288fa9a3543e9ea24194e6a5151abc8"}
Sep 10 16:30:16.512606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.512553    2577 scope.go:117] "RemoveContainer" containerID="fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"
Sep 10 16:30:16.522607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.522587    2577 scope.go:117] "RemoveContainer" containerID="fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"
Sep 10 16:30:16.522899 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:16.522883    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449\": container with ID starting with fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449 not found: ID does not exist" containerID="fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"
Sep 10 16:30:16.522939 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.522908    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449"} err="failed to get container status \"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449\": rpc error: code = NotFound desc = could not find container \"fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449\": container with ID starting with fea9ef5cc7faa6f9aa4274ffc569495710ada5e9042d7eb1bce8818b44a4f449 not found: ID does not exist"
Sep 10 16:30:16.537206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.537178    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:30:16.540974 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:16.540951    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-25d77f0b-5677f659f9-m79pv"]
Sep 10 16:30:18.113675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:18.113639    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" path="/var/lib/kubelet/pods/6e06e127-0bba-40ba-abc4-5393d62d4a0f/volumes"
Sep 10 16:30:20.656735 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.656701    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:20.657172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.657158    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" containerName="maas-api"
Sep 10 16:30:20.657218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.657174    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" containerName="maas-api"
Sep 10 16:30:20.657271 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.657261    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e06e127-0bba-40ba-abc4-5393d62d4a0f" containerName="maas-api"
Sep 10 16:30:20.661756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.661740    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.664667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.664643    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-2238d044-serving-cert\""
Sep 10 16:30:20.674090 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.674065    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:20.803633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.803595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjhnl\" (UniqueName: \"kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.803820 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.803642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.803820 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.803661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.803960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.803810    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.904698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.904654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.904877 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.904776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjhnl\" (UniqueName: \"kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.904877 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.904818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.904877 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.904848    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.904986 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:20.904966    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-serving-cert: secret "maas-api-e2e-migrate-2238d044-serving-cert" not found
Sep 10 16:30:20.905044 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:20.905034    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls podName:efa01e4e-75bb-4f34-b80e-47cd1aa7068b nodeName:}" failed. No retries permitted until 2026-09-10 16:30:21.405012436 +0000 UTC m=+793.806813547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls") pod "maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" (UID: "efa01e4e-75bb-4f34-b80e-47cd1aa7068b") : secret "maas-api-e2e-migrate-2238d044-serving-cert" not found
Sep 10 16:30:20.905447 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.905418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.907284 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.907227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:20.917998 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:20.917974    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjhnl\" (UniqueName: \"kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:21.410452 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:21.410411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:21.412971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:21.412944    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") pod \"maas-api-e2e-migrate-2238d044-644496c76b-v2vx4\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:21.572343 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:21.572306    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:21.854924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:21.854893    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:21.856014 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:30:21.855989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefa01e4e_75bb_4f34_b80e_47cd1aa7068b.slice/crio-d75479872e013942f1f5f8416e487a64c3a362e3430fea3189ad574c9cec87d9 WatchSource:0}: Error finding container d75479872e013942f1f5f8416e487a64c3a362e3430fea3189ad574c9cec87d9: Status 404 returned error can't find the container with id d75479872e013942f1f5f8416e487a64c3a362e3430fea3189ad574c9cec87d9
Sep 10 16:30:22.552231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:22.552192    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" event={"ID":"efa01e4e-75bb-4f34-b80e-47cd1aa7068b","Type":"ContainerStarted","Data":"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"}
Sep 10 16:30:22.552420 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:22.552258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" event={"ID":"efa01e4e-75bb-4f34-b80e-47cd1aa7068b","Type":"ContainerStarted","Data":"d75479872e013942f1f5f8416e487a64c3a362e3430fea3189ad574c9cec87d9"}
Sep 10 16:30:22.552420 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:22.552301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:22.571051 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:22.570990    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" podStartSLOduration=2.030418471 podStartE2EDuration="2.570970706s" totalImagesPullingTime="540.552235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:30:22.570598083 +0000 UTC m=+794.972399211" watchObservedRunningTime="2026-09-10 16:30:22.570970706 +0000 UTC m=+794.972771833"
Sep 10 16:30:28.563596 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:28.563569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:51.310625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.310589    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:51.311121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.310830    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" podUID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" containerName="maas-api" containerID="cri-o://1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406" gracePeriod=30
Sep 10 16:30:51.573849 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.573778    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:51.590778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.590749    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjhnl\" (UniqueName: \"kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl\") pod \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") "
Sep 10 16:30:51.590954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.590834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") pod \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") "
Sep 10 16:30:51.590954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.590876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca\") pod \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") "
Sep 10 16:30:51.590954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.590938    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs\") pod \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\" (UID: \"efa01e4e-75bb-4f34-b80e-47cd1aa7068b\") "
Sep 10 16:30:51.591410 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.591376    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "efa01e4e-75bb-4f34-b80e-47cd1aa7068b" (UID: "efa01e4e-75bb-4f34-b80e-47cd1aa7068b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:30:51.593227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.593200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl" (OuterVolumeSpecName: "kube-api-access-vjhnl") pod "efa01e4e-75bb-4f34-b80e-47cd1aa7068b" (UID: "efa01e4e-75bb-4f34-b80e-47cd1aa7068b"). InnerVolumeSpecName "kube-api-access-vjhnl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:30:51.593418 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.593393    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "efa01e4e-75bb-4f34-b80e-47cd1aa7068b" (UID: "efa01e4e-75bb-4f34-b80e-47cd1aa7068b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:51.593576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.593546    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "efa01e4e-75bb-4f34-b80e-47cd1aa7068b" (UID: "efa01e4e-75bb-4f34-b80e-47cd1aa7068b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:30:51.692156 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.692119    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:51.692373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.692176    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:51.692373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.692192    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjhnl\" (UniqueName: \"kubernetes.io/projected/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-kube-api-access-vjhnl\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:51.692373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.692205    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/efa01e4e-75bb-4f34-b80e-47cd1aa7068b-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:30:51.732950 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.732915    2577 generic.go:356] "Generic (PLEG): container finished" podID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" containerID="1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406" exitCode=0
Sep 10 16:30:51.733115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.732959    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"
Sep 10 16:30:51.733115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.732997    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" event={"ID":"efa01e4e-75bb-4f34-b80e-47cd1aa7068b","Type":"ContainerDied","Data":"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"}
Sep 10 16:30:51.733115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.733053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4" event={"ID":"efa01e4e-75bb-4f34-b80e-47cd1aa7068b","Type":"ContainerDied","Data":"d75479872e013942f1f5f8416e487a64c3a362e3430fea3189ad574c9cec87d9"}
Sep 10 16:30:51.733115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.733083    2577 scope.go:117] "RemoveContainer" containerID="1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"
Sep 10 16:30:51.743463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.743440    2577 scope.go:117] "RemoveContainer" containerID="1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"
Sep 10 16:30:51.743738 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:51.743719    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406\": container with ID starting with 1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406 not found: ID does not exist" containerID="1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"
Sep 10 16:30:51.743790 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.743746    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406"} err="failed to get container status \"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406\": rpc error: code = NotFound desc = could not find container \"1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406\": container with ID starting with 1cde5d502f5d0eb94bcdbf98abba3ba97d821be2d6e5d6c8bcfa24b9e8f9d406 not found: ID does not exist"
Sep 10 16:30:51.760340 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.760313    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:51.763942 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:51.763918    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-2238d044-644496c76b-v2vx4"]
Sep 10 16:30:52.113525 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:52.113494    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" path="/var/lib/kubelet/pods/efa01e4e-75bb-4f34-b80e-47cd1aa7068b/volumes"
Sep 10 16:30:58.613615 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.613566    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:30:58.614269 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.614241    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" containerName="maas-api"
Sep 10 16:30:58.614269 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.614262    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" containerName="maas-api"
Sep 10 16:30:58.614481 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.614388    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efa01e4e-75bb-4f34-b80e-47cd1aa7068b" containerName="maas-api"
Sep 10 16:30:58.618210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.618188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.621475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.621450    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f89c54f2-serving-cert\""
Sep 10 16:30:58.631733 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.631204    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:30:58.657947 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.657909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.658113 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.657965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.658113 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.658015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dz8j\" (UniqueName: \"kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.658239 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.658170    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.759291 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.759240    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.759489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.759364    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.759489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.759402    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.759489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.759446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dz8j\" (UniqueName: \"kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.759676 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:58.759533    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-serving-cert: secret "maas-api-e2e-ait-f89c54f2-serving-cert" not found
Sep 10 16:30:58.759676 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:30:58.759632    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls podName:13508709-a62c-4e22-b74e-71acffe50866 nodeName:}" failed. No retries permitted until 2026-09-10 16:30:59.259607717 +0000 UTC m=+831.661408829 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls") pod "maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" (UID: "13508709-a62c-4e22-b74e-71acffe50866") : secret "maas-api-e2e-ait-f89c54f2-serving-cert" not found
Sep 10 16:30:58.760120 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.760093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.761816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.761793    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:58.768825 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:58.768800    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dz8j\" (UniqueName: \"kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:59.264583 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:59.264549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:59.266933 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:59.266907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") pod \"maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:59.532669 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:59.532586    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:30:59.815902 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:30:59.815873    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:30:59.817443 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:30:59.817418    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13508709_a62c_4e22_b74e_71acffe50866.slice/crio-36fe5d8ea32ea85498b927cea25ad3716f213e05c102105414f782637f47ef0d WatchSource:0}: Error finding container 36fe5d8ea32ea85498b927cea25ad3716f213e05c102105414f782637f47ef0d: Status 404 returned error can't find the container with id 36fe5d8ea32ea85498b927cea25ad3716f213e05c102105414f782637f47ef0d
Sep 10 16:31:00.793765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:00.793730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" event={"ID":"13508709-a62c-4e22-b74e-71acffe50866","Type":"ContainerStarted","Data":"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"}
Sep 10 16:31:00.793938 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:00.793784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" event={"ID":"13508709-a62c-4e22-b74e-71acffe50866","Type":"ContainerStarted","Data":"36fe5d8ea32ea85498b927cea25ad3716f213e05c102105414f782637f47ef0d"}
Sep 10 16:31:00.793938 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:00.793822    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:31:00.814726 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:00.814682    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" podStartSLOduration=2.3751422890000002 podStartE2EDuration="2.814668646s" totalImagesPullingTime="439.526357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:30:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:31:00.812343488 +0000 UTC m=+833.214144611" watchObservedRunningTime="2026-09-10 16:31:00.814668646 +0000 UTC m=+833.216469772"
Sep 10 16:31:06.806178 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:06.806147    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:31:37.005815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.005784    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:31:37.006419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.006006    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" podUID="13508709-a62c-4e22-b74e-71acffe50866" containerName="maas-api" containerID="cri-o://c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c" gracePeriod=30
Sep 10 16:31:37.290585 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.290557    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:31:37.323558 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.323525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca\") pod \"13508709-a62c-4e22-b74e-71acffe50866\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") "
Sep 10 16:31:37.323720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.323589    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") pod \"13508709-a62c-4e22-b74e-71acffe50866\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") "
Sep 10 16:31:37.323720 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.323614    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs\") pod \"13508709-a62c-4e22-b74e-71acffe50866\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") "
Sep 10 16:31:37.323851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.323719    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dz8j\" (UniqueName: \"kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j\") pod \"13508709-a62c-4e22-b74e-71acffe50866\" (UID: \"13508709-a62c-4e22-b74e-71acffe50866\") "
Sep 10 16:31:37.324268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.324237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "13508709-a62c-4e22-b74e-71acffe50866" (UID: "13508709-a62c-4e22-b74e-71acffe50866"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:31:37.326507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.326436    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "13508709-a62c-4e22-b74e-71acffe50866" (UID: "13508709-a62c-4e22-b74e-71acffe50866"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:37.326507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.326458    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "13508709-a62c-4e22-b74e-71acffe50866" (UID: "13508709-a62c-4e22-b74e-71acffe50866"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:37.326816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.326783    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j" (OuterVolumeSpecName: "kube-api-access-6dz8j") pod "13508709-a62c-4e22-b74e-71acffe50866" (UID: "13508709-a62c-4e22-b74e-71acffe50866"). InnerVolumeSpecName "kube-api-access-6dz8j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:31:37.424499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.424472    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:37.424499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.424497    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/13508709-a62c-4e22-b74e-71acffe50866-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:37.424499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.424506    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dz8j\" (UniqueName: \"kubernetes.io/projected/13508709-a62c-4e22-b74e-71acffe50866-kube-api-access-6dz8j\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:37.424729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:37.424515    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/13508709-a62c-4e22-b74e-71acffe50866-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:38.027622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.027583    2577 generic.go:356] "Generic (PLEG): container finished" podID="13508709-a62c-4e22-b74e-71acffe50866" containerID="c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c" exitCode=0
Sep 10 16:31:38.028060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.027638    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"
Sep 10 16:31:38.028060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.027660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" event={"ID":"13508709-a62c-4e22-b74e-71acffe50866","Type":"ContainerDied","Data":"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"}
Sep 10 16:31:38.028060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.027742    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb" event={"ID":"13508709-a62c-4e22-b74e-71acffe50866","Type":"ContainerDied","Data":"36fe5d8ea32ea85498b927cea25ad3716f213e05c102105414f782637f47ef0d"}
Sep 10 16:31:38.028060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.027765    2577 scope.go:117] "RemoveContainer" containerID="c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"
Sep 10 16:31:38.037897 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.037881    2577 scope.go:117] "RemoveContainer" containerID="c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"
Sep 10 16:31:38.038183 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:31:38.038161    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c\": container with ID starting with c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c not found: ID does not exist" containerID="c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"
Sep 10 16:31:38.038302 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.038188    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c"} err="failed to get container status \"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c\": rpc error: code = NotFound desc = could not find container \"c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c\": container with ID starting with c8c3bc6272a0d127c681bb4775514fc28bd152ed548ffe057498b4794996375c not found: ID does not exist"
Sep 10 16:31:38.050905 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.050881    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:31:38.054910 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.054888    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f89c54f2-5c9b7b7c5f-jpfxb"]
Sep 10 16:31:38.114301 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:38.114257    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13508709-a62c-4e22-b74e-71acffe50866" path="/var/lib/kubelet/pods/13508709-a62c-4e22-b74e-71acffe50866/volumes"
Sep 10 16:31:44.760677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.760644    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:31:44.761127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.761078    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13508709-a62c-4e22-b74e-71acffe50866" containerName="maas-api"
Sep 10 16:31:44.761127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.761090    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="13508709-a62c-4e22-b74e-71acffe50866" containerName="maas-api"
Sep 10 16:31:44.761237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.761177    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="13508709-a62c-4e22-b74e-71acffe50866" containerName="maas-api"
Sep 10 16:31:44.765700 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.765682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.768899 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.768874    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-f39d5ea4-serving-cert\""
Sep 10 16:31:44.779169 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.779139    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:31:44.794802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.794772    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.794958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.794924    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.795030 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.795007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw46z\" (UniqueName: \"kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.795089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.795070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.896165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.896118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.896372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.896249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.896372 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:31:44.896269    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-serving-cert: secret "maas-api-e2e-derive-f39d5ea4-serving-cert" not found
Sep 10 16:31:44.896372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.896311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw46z\" (UniqueName: \"kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.896372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.896356    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.896607 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:31:44.896387    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls podName:948c19b8-5be6-4331-838b-2f7b6c3a4b2c nodeName:}" failed. No retries permitted until 2026-09-10 16:31:45.396362835 +0000 UTC m=+877.798163955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls") pod "maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" (UID: "948c19b8-5be6-4331-838b-2f7b6c3a4b2c") : secret "maas-api-e2e-derive-f39d5ea4-serving-cert" not found
Sep 10 16:31:44.896975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.896955    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.898878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.898855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:44.906545 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:44.906521    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw46z\" (UniqueName: \"kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:45.400165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:45.400128    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:45.402473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:45.402449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") pod \"maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:45.679058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:45.678971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:45.959491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:45.959465    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:31:45.961822 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:31:45.961792    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948c19b8_5be6_4331_838b_2f7b6c3a4b2c.slice/crio-53bc5451f7d59446884f07df203fcb6c26aa4a96d2daa8ef07c2edf15447f947 WatchSource:0}: Error finding container 53bc5451f7d59446884f07df203fcb6c26aa4a96d2daa8ef07c2edf15447f947: Status 404 returned error can't find the container with id 53bc5451f7d59446884f07df203fcb6c26aa4a96d2daa8ef07c2edf15447f947
Sep 10 16:31:46.079370 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:46.079332    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" event={"ID":"948c19b8-5be6-4331-838b-2f7b6c3a4b2c","Type":"ContainerStarted","Data":"53bc5451f7d59446884f07df203fcb6c26aa4a96d2daa8ef07c2edf15447f947"}
Sep 10 16:31:47.088856 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:47.088821    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" event={"ID":"948c19b8-5be6-4331-838b-2f7b6c3a4b2c","Type":"ContainerStarted","Data":"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"}
Sep 10 16:31:47.089263 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:47.089021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:31:47.113717 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:47.113659    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" podStartSLOduration=2.733735435 podStartE2EDuration="3.113643308s" totalImagesPullingTime="379.907873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:31:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:31:47.11057644 +0000 UTC m=+879.512377573" watchObservedRunningTime="2026-09-10 16:31:47.113643308 +0000 UTC m=+879.515444434"
Sep 10 16:31:47.967808 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:47.967772    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:31:47.968076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:47.968042    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerName="maas-api" containerID="cri-o://8b9251e814c8f68068186b5c0a8e7d30aeb441e705ab1407d1439246f3c26725" gracePeriod=30
Sep 10 16:31:48.027415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.027374    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:31:48.027647 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.027616    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerName="payload-pre-processing" containerID="cri-o://2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe" gracePeriod=30
Sep 10 16:31:48.098634 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.098604    2577 generic.go:356] "Generic (PLEG): container finished" podID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerID="8b9251e814c8f68068186b5c0a8e7d30aeb441e705ab1407d1439246f3c26725" exitCode=0
Sep 10 16:31:48.099159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.098688    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" event={"ID":"e83d586b-db35-4f7b-b419-b865e0d5cf96","Type":"ContainerDied","Data":"8b9251e814c8f68068186b5c0a8e7d30aeb441e705ab1407d1439246f3c26725"}
Sep 10 16:31:48.289343 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.289318    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:31:48.330181 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330151    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:31:48.330532 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca\") pod \"e83d586b-db35-4f7b-b419-b865e0d5cf96\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") "
Sep 10 16:31:48.330644 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") pod \"e83d586b-db35-4f7b-b419-b865e0d5cf96\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") "
Sep 10 16:31:48.330644 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs\") pod \"e83d586b-db35-4f7b-b419-b865e0d5cf96\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") "
Sep 10 16:31:48.330755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dzm6\" (UniqueName: \"kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6\") pod \"e83d586b-db35-4f7b-b419-b865e0d5cf96\" (UID: \"e83d586b-db35-4f7b-b419-b865e0d5cf96\") "
Sep 10 16:31:48.330944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.330887    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e83d586b-db35-4f7b-b419-b865e0d5cf96" (UID: "e83d586b-db35-4f7b-b419-b865e0d5cf96"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:31:48.331071 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.331060    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e83d586b-db35-4f7b-b419-b865e0d5cf96-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:48.332904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.332880    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e83d586b-db35-4f7b-b419-b865e0d5cf96" (UID: "e83d586b-db35-4f7b-b419-b865e0d5cf96"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:48.333112 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.333083    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e83d586b-db35-4f7b-b419-b865e0d5cf96" (UID: "e83d586b-db35-4f7b-b419-b865e0d5cf96"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:31:48.333317 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.333294    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6" (OuterVolumeSpecName: "kube-api-access-2dzm6") pod "e83d586b-db35-4f7b-b419-b865e0d5cf96" (UID: "e83d586b-db35-4f7b-b419-b865e0d5cf96"). InnerVolumeSpecName "kube-api-access-2dzm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:31:48.431994 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.431960    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glxxm\" (UniqueName: \"kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm\") pod \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") "
Sep 10 16:31:48.432183 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.432024    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume\") pod \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\" (UID: \"48ed2ae9-01a4-4388-af0b-b5d7288c03c0\") "
Sep 10 16:31:48.432375 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.432356    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:48.432450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.432378    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e83d586b-db35-4f7b-b419-b865e0d5cf96-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:48.432450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.432384    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "48ed2ae9-01a4-4388-af0b-b5d7288c03c0" (UID: "48ed2ae9-01a4-4388-af0b-b5d7288c03c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:31:48.432450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.432392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dzm6\" (UniqueName: \"kubernetes.io/projected/e83d586b-db35-4f7b-b419-b865e0d5cf96-kube-api-access-2dzm6\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:48.433975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.433955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm" (OuterVolumeSpecName: "kube-api-access-glxxm") pod "48ed2ae9-01a4-4388-af0b-b5d7288c03c0" (UID: "48ed2ae9-01a4-4388-af0b-b5d7288c03c0"). InnerVolumeSpecName "kube-api-access-glxxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:31:48.533386 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.533258    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glxxm\" (UniqueName: \"kubernetes.io/projected/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-kube-api-access-glxxm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:48.533386 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:48.533332    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed2ae9-01a4-4388-af0b-b5d7288c03c0-plugins-config-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:31:49.107635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.107607    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"
Sep 10 16:31:49.107635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.107626    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" event={"ID":"e83d586b-db35-4f7b-b419-b865e0d5cf96","Type":"ContainerDied","Data":"c48b502aabe2880e6453fd45e02385387b39f5deffdc064cd13138e58d54d765"}
Sep 10 16:31:49.108172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.107677    2577 scope.go:117] "RemoveContainer" containerID="8b9251e814c8f68068186b5c0a8e7d30aeb441e705ab1407d1439246f3c26725"
Sep 10 16:31:49.109176 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.109155    2577 generic.go:356] "Generic (PLEG): container finished" podID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerID="2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe" exitCode=0
Sep 10 16:31:49.109259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.109209    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"
Sep 10 16:31:49.109259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.109239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" event={"ID":"48ed2ae9-01a4-4388-af0b-b5d7288c03c0","Type":"ContainerDied","Data":"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"}
Sep 10 16:31:49.109382 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.109310    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" event={"ID":"48ed2ae9-01a4-4388-af0b-b5d7288c03c0","Type":"ContainerDied","Data":"08e8d00472d41816beaad7fd7705215d79ac471e1c446ba344841f438187fd39"}
Sep 10 16:31:49.119563 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.119541    2577 scope.go:117] "RemoveContainer" containerID="2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"
Sep 10 16:31:49.130585 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.130521    2577 scope.go:117] "RemoveContainer" containerID="2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"
Sep 10 16:31:49.130859 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:31:49.130837    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe\": container with ID starting with 2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe not found: ID does not exist" containerID="2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"
Sep 10 16:31:49.130952 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.130867    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe"} err="failed to get container status \"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe\": rpc error: code = NotFound desc = could not find container \"2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe\": container with ID starting with 2917ba9233402def261cac6f717723f593321322bce87d2d3885fdf486ae93fe not found: ID does not exist"
Sep 10 16:31:49.133337 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.133317    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:31:49.138868 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.138848    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd"]
Sep 10 16:31:49.150192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.150173    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:31:49.154235 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.154216    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh"]
Sep 10 16:31:49.248399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:49.248315    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-autores-2d818c-5d7b4b66f4-cmxmh" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.69:9004: i/o timeout"
Sep 10 16:31:50.113060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:50.113029    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" path="/var/lib/kubelet/pods/48ed2ae9-01a4-4388-af0b-b5d7288c03c0/volumes"
Sep 10 16:31:50.113516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:50.113499    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" path="/var/lib/kubelet/pods/e83d586b-db35-4f7b-b419-b865e0d5cf96/volumes"
Sep 10 16:31:51.247999 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:51.247954    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-autores-2d818c-769f7cd56c-mjdrd" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.68:8443/health\": context deadline exceeded"
Sep 10 16:31:53.104904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:31:53.104875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:32:03.046231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.046137    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:32:03.047592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047562    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerName="maas-api"
Sep 10 16:32:03.047592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047592    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerName="maas-api"
Sep 10 16:32:03.047823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047632    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerName="payload-pre-processing"
Sep 10 16:32:03.047823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047641    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerName="payload-pre-processing"
Sep 10 16:32:03.047823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047759    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e83d586b-db35-4f7b-b419-b865e0d5cf96" containerName="maas-api"
Sep 10 16:32:03.047823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.047772    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="48ed2ae9-01a4-4388-af0b-b5d7288c03c0" containerName="payload-pre-processing"
Sep 10 16:32:03.053118 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.053094    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.055870 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.055830    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-09da25-serving-cert\""
Sep 10 16:32:03.063500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.063461    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:32:03.177980 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.177941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw7ts\" (UniqueName: \"kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.177980 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.177985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.178218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.178007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.178218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.178090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.279401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.279357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw7ts\" (UniqueName: \"kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.279606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.279411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.279606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.279446    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.279606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.279514    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.279606 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:32:03.279588    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-serving-cert: secret "maas-api-e2e-explicit-09da25-serving-cert" not found
Sep 10 16:32:03.279831 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:32:03.279690    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls podName:46885803-0be3-4599-9f9f-aeff42891e76 nodeName:}" failed. No retries permitted until 2026-09-10 16:32:03.779666964 +0000 UTC m=+896.181468089 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls") pod "maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" (UID: "46885803-0be3-4599-9f9f-aeff42891e76") : secret "maas-api-e2e-explicit-09da25-serving-cert" not found
Sep 10 16:32:03.280210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.280188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.282079 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.282056    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.293211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.293186    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw7ts\" (UniqueName: \"kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.784657 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.784617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.787382 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.787349    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") pod \"maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:03.985183 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:03.985137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:04.263612 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:04.263578    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:32:04.265484 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:32:04.265457    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46885803_0be3_4599_9f9f_aeff42891e76.slice/crio-101c9cd87637b75364eb3802287ccbe55c954ad2484255a6fd3dc21b9c7734f0 WatchSource:0}: Error finding container 101c9cd87637b75364eb3802287ccbe55c954ad2484255a6fd3dc21b9c7734f0: Status 404 returned error can't find the container with id 101c9cd87637b75364eb3802287ccbe55c954ad2484255a6fd3dc21b9c7734f0
Sep 10 16:32:05.213122 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:05.213082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" event={"ID":"46885803-0be3-4599-9f9f-aeff42891e76","Type":"ContainerStarted","Data":"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"}
Sep 10 16:32:05.213342 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:05.213154    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:05.213342 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:05.213177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" event={"ID":"46885803-0be3-4599-9f9f-aeff42891e76","Type":"ContainerStarted","Data":"101c9cd87637b75364eb3802287ccbe55c954ad2484255a6fd3dc21b9c7734f0"}
Sep 10 16:32:05.235380 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:05.235327    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" podStartSLOduration=1.7510399460000001 podStartE2EDuration="2.235312059s" totalImagesPullingTime="484.272113ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:32:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:32:05.231746716 +0000 UTC m=+897.633547844" watchObservedRunningTime="2026-09-10 16:32:05.235312059 +0000 UTC m=+897.637113186"
Sep 10 16:32:08.106832 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:08.106800    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:32:08.108830 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:08.108806    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:32:11.226858 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:11.226824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:32:11.976570 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:11.976539    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:32:11.976865 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:11.976829    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" podUID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" containerName="maas-api" containerID="cri-o://102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c" gracePeriod=30
Sep 10 16:32:12.251115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.251088    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:32:12.254106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.254080    2577 generic.go:356] "Generic (PLEG): container finished" podID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" containerID="102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c" exitCode=0
Sep 10 16:32:12.254232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.254126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"
Sep 10 16:32:12.254232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.254159    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" event={"ID":"948c19b8-5be6-4331-838b-2f7b6c3a4b2c","Type":"ContainerDied","Data":"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"}
Sep 10 16:32:12.254232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.254214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj" event={"ID":"948c19b8-5be6-4331-838b-2f7b6c3a4b2c","Type":"ContainerDied","Data":"53bc5451f7d59446884f07df203fcb6c26aa4a96d2daa8ef07c2edf15447f947"}
Sep 10 16:32:12.254377 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.254242    2577 scope.go:117] "RemoveContainer" containerID="102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"
Sep 10 16:32:12.262435 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.262413    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs\") pod \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") "
Sep 10 16:32:12.262562 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.262466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dw46z\" (UniqueName: \"kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z\") pod \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") "
Sep 10 16:32:12.262624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.262571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") pod \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") "
Sep 10 16:32:12.262684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.262632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca\") pod \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\" (UID: \"948c19b8-5be6-4331-838b-2f7b6c3a4b2c\") "
Sep 10 16:32:12.263055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.263030    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "948c19b8-5be6-4331-838b-2f7b6c3a4b2c" (UID: "948c19b8-5be6-4331-838b-2f7b6c3a4b2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:32:12.264371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.264342    2577 scope.go:117] "RemoveContainer" containerID="102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"
Sep 10 16:32:12.264512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.264491    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "948c19b8-5be6-4331-838b-2f7b6c3a4b2c" (UID: "948c19b8-5be6-4331-838b-2f7b6c3a4b2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:32:12.264656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.264637    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "948c19b8-5be6-4331-838b-2f7b6c3a4b2c" (UID: "948c19b8-5be6-4331-838b-2f7b6c3a4b2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:32:12.264726 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:32:12.264691    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c\": container with ID starting with 102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c not found: ID does not exist" containerID="102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"
Sep 10 16:32:12.264726 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.264713    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c"} err="failed to get container status \"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c\": rpc error: code = NotFound desc = could not find container \"102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c\": container with ID starting with 102cbeeb39b3dce5462aadd408ab90825a9a34191537062ed3b7d17f65bd8f0c not found: ID does not exist"
Sep 10 16:32:12.264884 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.264865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z" (OuterVolumeSpecName: "kube-api-access-dw46z") pod "948c19b8-5be6-4331-838b-2f7b6c3a4b2c" (UID: "948c19b8-5be6-4331-838b-2f7b6c3a4b2c"). InnerVolumeSpecName "kube-api-access-dw46z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:32:12.363638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.363604    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:32:12.363638 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.363636    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dw46z\" (UniqueName: \"kubernetes.io/projected/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-kube-api-access-dw46z\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:32:12.363889 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.363648    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:32:12.363889 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.363656    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/948c19b8-5be6-4331-838b-2f7b6c3a4b2c-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:32:12.592261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.592231    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:32:12.594987 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:12.594960    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-f39d5ea4-67948bb7f8-rxwxj"]
Sep 10 16:32:14.121427 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:32:14.121390    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" path="/var/lib/kubelet/pods/948c19b8-5be6-4331-838b-2f7b6c3a4b2c/volumes"
Sep 10 16:33:03.465108 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.465074    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:33:03.466439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.466390    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" podUID="46885803-0be3-4599-9f9f-aeff42891e76" containerName="maas-api" containerID="cri-o://bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525" gracePeriod=30
Sep 10 16:33:03.731355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.731291    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:33:03.746150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.746119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fw7ts\" (UniqueName: \"kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts\") pod \"46885803-0be3-4599-9f9f-aeff42891e76\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") "
Sep 10 16:33:03.746150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.746151    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs\") pod \"46885803-0be3-4599-9f9f-aeff42891e76\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") "
Sep 10 16:33:03.746408 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.746228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca\") pod \"46885803-0be3-4599-9f9f-aeff42891e76\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") "
Sep 10 16:33:03.746408 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.746251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") pod \"46885803-0be3-4599-9f9f-aeff42891e76\" (UID: \"46885803-0be3-4599-9f9f-aeff42891e76\") "
Sep 10 16:33:03.746628 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.746601    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46885803-0be3-4599-9f9f-aeff42891e76" (UID: "46885803-0be3-4599-9f9f-aeff42891e76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:33:03.748595 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.748561    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46885803-0be3-4599-9f9f-aeff42891e76" (UID: "46885803-0be3-4599-9f9f-aeff42891e76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:03.748756 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.748626    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts" (OuterVolumeSpecName: "kube-api-access-fw7ts") pod "46885803-0be3-4599-9f9f-aeff42891e76" (UID: "46885803-0be3-4599-9f9f-aeff42891e76"). InnerVolumeSpecName "kube-api-access-fw7ts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:33:03.748870 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.748811    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46885803-0be3-4599-9f9f-aeff42891e76" (UID: "46885803-0be3-4599-9f9f-aeff42891e76"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:03.847393 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.847358    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46885803-0be3-4599-9f9f-aeff42891e76-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:03.847393 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.847387    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:03.847393 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.847397    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fw7ts\" (UniqueName: \"kubernetes.io/projected/46885803-0be3-4599-9f9f-aeff42891e76-kube-api-access-fw7ts\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:03.847635 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:03.847405    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46885803-0be3-4599-9f9f-aeff42891e76-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:04.568885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.568799    2577 generic.go:356] "Generic (PLEG): container finished" podID="46885803-0be3-4599-9f9f-aeff42891e76" containerID="bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525" exitCode=0
Sep 10 16:33:04.568885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.568846    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"
Sep 10 16:33:04.569350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.568837    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" event={"ID":"46885803-0be3-4599-9f9f-aeff42891e76","Type":"ContainerDied","Data":"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"}
Sep 10 16:33:04.569350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.568964    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt" event={"ID":"46885803-0be3-4599-9f9f-aeff42891e76","Type":"ContainerDied","Data":"101c9cd87637b75364eb3802287ccbe55c954ad2484255a6fd3dc21b9c7734f0"}
Sep 10 16:33:04.569350 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.568990    2577 scope.go:117] "RemoveContainer" containerID="bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"
Sep 10 16:33:04.578854 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.578825    2577 scope.go:117] "RemoveContainer" containerID="bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"
Sep 10 16:33:04.579107 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:33:04.579085    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525\": container with ID starting with bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525 not found: ID does not exist" containerID="bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"
Sep 10 16:33:04.579174 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.579113    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525"} err="failed to get container status \"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525\": rpc error: code = NotFound desc = could not find container \"bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525\": container with ID starting with bd096084074e4dc0807b84bf0b684f33415c8ce1f961d6bbc5faa2a68eb23525 not found: ID does not exist"
Sep 10 16:33:04.587982 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.587818    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:33:04.590798 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:04.590772    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-09da25-7b84fb747c-mvjtt"]
Sep 10 16:33:06.113579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:06.113547    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46885803-0be3-4599-9f9f-aeff42891e76" path="/var/lib/kubelet/pods/46885803-0be3-4599-9f9f-aeff42891e76/volumes"
Sep 10 16:33:56.261364 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261330    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:33:56.261834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261753    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46885803-0be3-4599-9f9f-aeff42891e76" containerName="maas-api"
Sep 10 16:33:56.261834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261771    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="46885803-0be3-4599-9f9f-aeff42891e76" containerName="maas-api"
Sep 10 16:33:56.261834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261783    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" containerName="maas-api"
Sep 10 16:33:56.261834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261788    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" containerName="maas-api"
Sep 10 16:33:56.261991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261858    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="46885803-0be3-4599-9f9f-aeff42891e76" containerName="maas-api"
Sep 10 16:33:56.261991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.261866    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="948c19b8-5be6-4331-838b-2f7b6c3a4b2c" containerName="maas-api"
Sep 10 16:33:56.265678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.265656    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.268648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.268627    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e6e77069-serving-cert\""
Sep 10 16:33:56.294818 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.294768    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:33:56.346288 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.346236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.346467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.346346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.346467 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.346453    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf2bg\" (UniqueName: \"kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.346559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.346504    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.447189 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.447156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.447391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.447217    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.447391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.447270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf2bg\" (UniqueName: \"kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.447391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.447336    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.447954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.447932    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.449755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.449733    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.449857 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.449777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.455011 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.454989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf2bg\" (UniqueName: \"kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg\") pod \"maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.594529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.594437    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:56.874629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.874601    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:33:56.876082 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:33:56.876043    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45c3aa19_228e_44fb_bdf0_d0a4c7f92e82.slice/crio-64dc856190c7688e670817cb4f6c6805d3ea27dc98f6dd24b6afd8bc64573cff WatchSource:0}: Error finding container 64dc856190c7688e670817cb4f6c6805d3ea27dc98f6dd24b6afd8bc64573cff: Status 404 returned error can't find the container with id 64dc856190c7688e670817cb4f6c6805d3ea27dc98f6dd24b6afd8bc64573cff
Sep 10 16:33:56.877750 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:56.877732    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:33:57.887380 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:57.887314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" event={"ID":"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82","Type":"ContainerStarted","Data":"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"}
Sep 10 16:33:57.887850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:57.887388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" event={"ID":"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82","Type":"ContainerStarted","Data":"64dc856190c7688e670817cb4f6c6805d3ea27dc98f6dd24b6afd8bc64573cff"}
Sep 10 16:33:57.887850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:57.887559    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:33:57.913165 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:57.913114    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" podStartSLOduration=1.388535606 podStartE2EDuration="1.913099342s" totalImagesPullingTime="524.563736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:33:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:33:57.911524517 +0000 UTC m=+1010.313325644" watchObservedRunningTime="2026-09-10 16:33:57.913099342 +0000 UTC m=+1010.314900470"
Sep 10 16:33:59.575504 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.575468    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:33:59.575992 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.575737    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" podUID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" containerName="maas-api" containerID="cri-o://dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2" gracePeriod=30
Sep 10 16:33:59.848177 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.848153    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:33:59.902260 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.902227    2577 generic.go:356] "Generic (PLEG): container finished" podID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" containerID="dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2" exitCode=0
Sep 10 16:33:59.902478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.902315    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"
Sep 10 16:33:59.902478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.902381    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" event={"ID":"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f","Type":"ContainerDied","Data":"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"}
Sep 10 16:33:59.902478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.902424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6" event={"ID":"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f","Type":"ContainerDied","Data":"f3e4686eedb97c377e3d328f810d32ef0355eca03625556ef13619d031cc478b"}
Sep 10 16:33:59.902478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.902448    2577 scope.go:117] "RemoveContainer" containerID="dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"
Sep 10 16:33:59.912803 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.912779    2577 scope.go:117] "RemoveContainer" containerID="dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"
Sep 10 16:33:59.913200 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:33:59.913169    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2\": container with ID starting with dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2 not found: ID does not exist" containerID="dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"
Sep 10 16:33:59.913341 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.913209    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2"} err="failed to get container status \"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2\": rpc error: code = NotFound desc = could not find container \"dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2\": container with ID starting with dc838f95b895342d8d37e820dd4a7bb2fe6d19c628a3e2cae7fc96af609efad2 not found: ID does not exist"
Sep 10 16:33:59.983599 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.983563    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjrmq\" (UniqueName: \"kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq\") pod \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") "
Sep 10 16:33:59.983844 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.983829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") pod \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") "
Sep 10 16:33:59.983995 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.983981    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs\") pod \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") "
Sep 10 16:33:59.984129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.984117    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca\") pod \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\" (UID: \"aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f\") "
Sep 10 16:33:59.984620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.984593    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" (UID: "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:33:59.987262 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.987229    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" (UID: "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:33:59.987498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.987473    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:59.987655 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.987639    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:33:59.990355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.989695    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq" (OuterVolumeSpecName: "kube-api-access-rjrmq") pod "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" (UID: "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f"). InnerVolumeSpecName "kube-api-access-rjrmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:33:59.990691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:33:59.990589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" (UID: "aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:00.089213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:00.089109    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:00.089213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:00.089144    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjrmq\" (UniqueName: \"kubernetes.io/projected/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f-kube-api-access-rjrmq\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:00.221145 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:00.221110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:34:00.224009 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:00.223983    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-1abdaf-76b75ff5f6-4hnn6"]
Sep 10 16:34:02.114661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:02.114622    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" path="/var/lib/kubelet/pods/aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f/volumes"
Sep 10 16:34:03.899605 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:03.899576    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:34:06.859827 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:06.859791    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:34:06.860332 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:06.860080    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" containerID="cri-o://be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33" gracePeriod=30
Sep 10 16:34:08.481156 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:08.481119    2577 patch_prober.go:28] interesting pod/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 16:34:08.481538 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:08.481185    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 16:34:12.121824 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.121791    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:34:12.301003 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.300920    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301003 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.300962    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301049    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k897l\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301078    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301107    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301200    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301246    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data\") pod \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\" (UID: \"7ce9f22d-f82a-44ab-b72c-b83ace64023d\") "
Sep 10 16:34:12.301499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:34:12.301499 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:34:12.301712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301607    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.301712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301627    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-workload-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.301712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301638    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data" (OuterVolumeSpecName: "istio-data") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:34:12.301712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301693    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:12.301712 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.301705    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:34:12.303180 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.303150    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 16:34:12.303422 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.303391    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:34:12.303661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.303638    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l" (OuterVolumeSpecName: "kube-api-access-k897l") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "kube-api-access-k897l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:12.303757 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.303658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token" (OuterVolumeSpecName: "istio-token") pod "7ce9f22d-f82a-44ab-b72c-b83ace64023d" (UID: "7ce9f22d-f82a-44ab-b72c-b83ace64023d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:12.402690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402639    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-envoy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402680    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-data\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402689    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-podinfo\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402697    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k897l\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-kube-api-access-k897l\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402690 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402707    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402715    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7ce9f22d-f82a-44ab-b72c-b83ace64023d-credential-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.402991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.402723    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7ce9f22d-f82a-44ab-b72c-b83ace64023d-istiod-ca-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:12.982113 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.982078    2577 generic.go:356] "Generic (PLEG): container finished" podID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerID="be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33" exitCode=0
Sep 10 16:34:12.982331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.982126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"
Sep 10 16:34:12.982331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.982166    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerDied","Data":"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"}
Sep 10 16:34:12.982331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.982224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc" event={"ID":"7ce9f22d-f82a-44ab-b72c-b83ace64023d","Type":"ContainerDied","Data":"d9caaff33495db9e07fb5cad27a2640339fc875d324da8d782b1f325fde5fda4"}
Sep 10 16:34:12.982331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.982255    2577 scope.go:117] "RemoveContainer" containerID="be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"
Sep 10 16:34:12.992718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:12.992698    2577 scope.go:117] "RemoveContainer" containerID="3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"
Sep 10 16:34:13.001063 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.001046    2577 scope.go:117] "RemoveContainer" containerID="be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"
Sep 10 16:34:13.001357 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:13.001330    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33\": container with ID starting with be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33 not found: ID does not exist" containerID="be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"
Sep 10 16:34:13.001429 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.001360    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33"} err="failed to get container status \"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33\": rpc error: code = NotFound desc = could not find container \"be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33\": container with ID starting with be412df4c2835d3890cff7e0ad732bd0b119e639b77c1146334429f77081ef33 not found: ID does not exist"
Sep 10 16:34:13.001429 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.001384    2577 scope.go:117] "RemoveContainer" containerID="3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"
Sep 10 16:34:13.001629 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:13.001613    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153\": container with ID starting with 3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153 not found: ID does not exist" containerID="3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"
Sep 10 16:34:13.001670 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.001633    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153"} err="failed to get container status \"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153\": rpc error: code = NotFound desc = could not find container \"3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153\": container with ID starting with 3e9462a7489c90421e6941cca57ae5b373753e188298842da9dce637172d3153 not found: ID does not exist"
Sep 10 16:34:13.011613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.011579    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:34:13.013537 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:13.013512    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1abdaf-openshift-default-85d9f9d488-dxhkc"]
Sep 10 16:34:14.113548 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:14.113509    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" path="/var/lib/kubelet/pods/7ce9f22d-f82a-44ab-b72c-b83ace64023d/volumes"
Sep 10 16:34:17.943562 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:17.943524    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:34:17.944052 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:17.943782    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" podUID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" containerName="maas-api" containerID="cri-o://0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2" gracePeriod=30
Sep 10 16:34:18.207583 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.207519    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:34:18.360974 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.360936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls\") pod \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") "
Sep 10 16:34:18.361172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.360987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb9pp\" (UniqueName: \"kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp\") pod \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") "
Sep 10 16:34:18.361172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.361023    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca\") pod \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") "
Sep 10 16:34:18.361172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.361053    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs\") pod \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\" (UID: \"c41f77fd-d833-459a-aea9-69ed25c1f3cb\") "
Sep 10 16:34:18.361488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.361456    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c41f77fd-d833-459a-aea9-69ed25c1f3cb" (UID: "c41f77fd-d833-459a-aea9-69ed25c1f3cb"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:18.363135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.363104    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp" (OuterVolumeSpecName: "kube-api-access-kb9pp") pod "c41f77fd-d833-459a-aea9-69ed25c1f3cb" (UID: "c41f77fd-d833-459a-aea9-69ed25c1f3cb"). InnerVolumeSpecName "kube-api-access-kb9pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:18.363232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.363125    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c41f77fd-d833-459a-aea9-69ed25c1f3cb" (UID: "c41f77fd-d833-459a-aea9-69ed25c1f3cb"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:18.363232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.363174    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c41f77fd-d833-459a-aea9-69ed25c1f3cb" (UID: "c41f77fd-d833-459a-aea9-69ed25c1f3cb"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:18.462433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.462344    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:18.462433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.462372    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb9pp\" (UniqueName: \"kubernetes.io/projected/c41f77fd-d833-459a-aea9-69ed25c1f3cb-kube-api-access-kb9pp\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:18.462433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.462382    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c41f77fd-d833-459a-aea9-69ed25c1f3cb-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:18.462433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:18.462390    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c41f77fd-d833-459a-aea9-69ed25c1f3cb-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:19.021755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.021721    2577 generic.go:356] "Generic (PLEG): container finished" podID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" containerID="0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2" exitCode=0
Sep 10 16:34:19.022152 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.021774    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"
Sep 10 16:34:19.022152 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.021804    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" event={"ID":"c41f77fd-d833-459a-aea9-69ed25c1f3cb","Type":"ContainerDied","Data":"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"}
Sep 10 16:34:19.022152 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.021854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt" event={"ID":"c41f77fd-d833-459a-aea9-69ed25c1f3cb","Type":"ContainerDied","Data":"ba01a3140651ae3fb32c0946fbd9f52597eaa3c18d1c30e3dc0a06b09eff3e44"}
Sep 10 16:34:19.022152 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.021876    2577 scope.go:117] "RemoveContainer" containerID="0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"
Sep 10 16:34:19.035200 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.035174    2577 scope.go:117] "RemoveContainer" containerID="0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"
Sep 10 16:34:19.035739 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:19.035706    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2\": container with ID starting with 0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2 not found: ID does not exist" containerID="0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"
Sep 10 16:34:19.035842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.035745    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2"} err="failed to get container status \"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2\": rpc error: code = NotFound desc = could not find container \"0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2\": container with ID starting with 0fad54a780979b32a044890f86e136d047ff88902ab8b1c3d73061da62ea33e2 not found: ID does not exist"
Sep 10 16:34:19.050206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.050177    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:34:19.054068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:19.054035    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-9f6082-65dd68796f-bkjpt"]
Sep 10 16:34:20.112880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:20.112837    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" path="/var/lib/kubelet/pods/c41f77fd-d833-459a-aea9-69ed25c1f3cb/volumes"
Sep 10 16:34:25.857031 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.856993    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:34:25.857619 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857602    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:25.857665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857622    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:25.857665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857644    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" containerName="maas-api"
Sep 10 16:34:25.857665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857652    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" containerName="maas-api"
Sep 10 16:34:25.857755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857672    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" containerName="maas-api"
Sep 10 16:34:25.857755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857681    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" containerName="maas-api"
Sep 10 16:34:25.857819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:25.857819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857790    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:25.857819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857798    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c41f77fd-d833-459a-aea9-69ed25c1f3cb" containerName="maas-api"
Sep 10 16:34:25.857819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.857805    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa9b9bcb-5f6e-4aa8-b38b-427f6911d85f" containerName="maas-api"
Sep 10 16:34:25.862717 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.862694    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.865597 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.865573    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-1925d8-openshift-default-dockercfg-m69m8\""
Sep 10 16:34:25.865597 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.865586    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 16:34:25.888105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.888059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:34:25.930410 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930421    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c4dc\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930715    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930891 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930804    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:25.930891 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:25.930828    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032299 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032231    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032363    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032500 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c4dc\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.032770 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032737    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.033023 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.032996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.033136 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.033021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.033462 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.033435    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.033581 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.033566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.034548 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.034526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.035157 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.035138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.040721 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.040696    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.041086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.041065    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c4dc\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc\") pod \"e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") " pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.175355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.175316    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:26.453826 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.453801    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:34:26.455229 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:34:26.455184    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeff30db2_8924_4ea2_91e5_93a0554eeece.slice/crio-a7ab4a97e7060f89c00d61b9d79e570988ca5476e8f4cec5639271216adc8a97 WatchSource:0}: Error finding container a7ab4a97e7060f89c00d61b9d79e570988ca5476e8f4cec5639271216adc8a97: Status 404 returned error can't find the container with id a7ab4a97e7060f89c00d61b9d79e570988ca5476e8f4cec5639271216adc8a97
Sep 10 16:34:26.457900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.457863    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:26.458006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.457951    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:26.458006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:26.457997    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:27.018224 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.018185    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:34:27.018807 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.018451    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" podUID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" containerName="maas-api" containerID="cri-o://3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4" gracePeriod=30
Sep 10 16:34:27.075825 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.075783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerStarted","Data":"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"}
Sep 10 16:34:27.076016 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.075861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerStarted","Data":"a7ab4a97e7060f89c00d61b9d79e570988ca5476e8f4cec5639271216adc8a97"}
Sep 10 16:34:27.102008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.101954    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podStartSLOduration=2.101935208 podStartE2EDuration="2.101935208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:34:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:34:27.099197184 +0000 UTC m=+1039.500998313" watchObservedRunningTime="2026-09-10 16:34:27.101935208 +0000 UTC m=+1039.503736335"
Sep 10 16:34:27.176151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.176116    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:27.182231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.182206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:27.271355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.271294    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:34:27.342502 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.342467    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf2bg\" (UniqueName: \"kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg\") pod \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") "
Sep 10 16:34:27.342692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.342538    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs\") pod \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") "
Sep 10 16:34:27.342692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.342571    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca\") pod \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") "
Sep 10 16:34:27.342692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.342605    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls\") pod \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\" (UID: \"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82\") "
Sep 10 16:34:27.342983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.342951    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" (UID: "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:34:27.344695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.344671    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" (UID: "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:27.344786 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.344716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" (UID: "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:34:27.344786 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.344778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg" (OuterVolumeSpecName: "kube-api-access-wf2bg") pod "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" (UID: "45c3aa19-228e-44fb-bdf0-d0a4c7f92e82"). InnerVolumeSpecName "kube-api-access-wf2bg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:34:27.444058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.444026    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf2bg\" (UniqueName: \"kubernetes.io/projected/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-kube-api-access-wf2bg\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:27.444058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.444055    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:27.444058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.444065    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:27.444312 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:27.444073    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:34:28.083783 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.083743    2577 generic.go:356] "Generic (PLEG): container finished" podID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" containerID="3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4" exitCode=0
Sep 10 16:34:28.083783 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.083791    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"
Sep 10 16:34:28.084394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.083815    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" event={"ID":"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82","Type":"ContainerDied","Data":"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"}
Sep 10 16:34:28.084394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.083885    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5" event={"ID":"45c3aa19-228e-44fb-bdf0-d0a4c7f92e82","Type":"ContainerDied","Data":"64dc856190c7688e670817cb4f6c6805d3ea27dc98f6dd24b6afd8bc64573cff"}
Sep 10 16:34:28.084394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.083917    2577 scope.go:117] "RemoveContainer" containerID="3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"
Sep 10 16:34:28.084555 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.084509    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:28.086448 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.086422    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:28.095501 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.095485    2577 scope.go:117] "RemoveContainer" containerID="3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"
Sep 10 16:34:28.095803 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:28.095784    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4\": container with ID starting with 3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4 not found: ID does not exist" containerID="3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"
Sep 10 16:34:28.095883 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.095814    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4"} err="failed to get container status \"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4\": rpc error: code = NotFound desc = could not find container \"3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4\": container with ID starting with 3f6881244d50d2179a548d6cc278f58242d8615b7da37e4e7f221763f47b99d4 not found: ID does not exist"
Sep 10 16:34:28.130540 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.130392    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:34:28.133214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:28.133189    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e6e77069-8466d4cff6-fqmt5"]
Sep 10 16:34:30.113915 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.113880    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" path="/var/lib/kubelet/pods/45c3aa19-228e-44fb-bdf0-d0a4c7f92e82/volumes"
Sep 10 16:34:30.280884 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.280847    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:34:30.281490 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.281467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:30.281490 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.281491    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ce9f22d-f82a-44ab-b72c-b83ace64023d" containerName="istio-proxy"
Sep 10 16:34:30.281666 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.281505    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" containerName="maas-api"
Sep 10 16:34:30.281666 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.281514    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" containerName="maas-api"
Sep 10 16:34:30.281666 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.281624    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="45c3aa19-228e-44fb-bdf0-d0a4c7f92e82" containerName="maas-api"
Sep 10 16:34:30.286872 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.286848    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.294952 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.294882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-1925d8-serving-cert\""
Sep 10 16:34:30.296920 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.296617    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:34:30.371974 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.371882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.371974 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.371922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb4qx\" (UniqueName: \"kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.372220 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.372049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.372220 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.372088    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.473230 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.473189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.473230 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.473233    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.473482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.473337    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.473482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.473357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb4qx\" (UniqueName: \"kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.473482 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:30.473411    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-serving-cert: secret "maas-api-e2e-shared-a-w3-1925d8-serving-cert" not found
Sep 10 16:34:30.473603 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:34:30.473493    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls podName:bfba5bce-2950-4220-840d-759aeb103c54 nodeName:}" failed. No retries permitted until 2026-09-10 16:34:30.973471541 +0000 UTC m=+1043.375272674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls") pod "maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" (UID: "bfba5bce-2950-4220-840d-759aeb103c54") : secret "maas-api-e2e-shared-a-w3-1925d8-serving-cert" not found
Sep 10 16:34:30.473978 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.473958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.475781 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.475763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.482031 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.482006    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb4qx\" (UniqueName: \"kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.978625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.978587    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:30.981056 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:30.981029    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:31.202659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:31.202587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:31.480642 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:31.480611    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:34:31.481929 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:34:31.481898    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfba5bce_2950_4220_840d_759aeb103c54.slice/crio-ea114ceafd4325c1e057090e1db1e6dcb5f842b9cf0d56197b03073ed5a08b71 WatchSource:0}: Error finding container ea114ceafd4325c1e057090e1db1e6dcb5f842b9cf0d56197b03073ed5a08b71: Status 404 returned error can't find the container with id ea114ceafd4325c1e057090e1db1e6dcb5f842b9cf0d56197b03073ed5a08b71
Sep 10 16:34:32.112573 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:32.112553    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:32.112715 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:32.112597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" event={"ID":"bfba5bce-2950-4220-840d-759aeb103c54","Type":"ContainerStarted","Data":"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"}
Sep 10 16:34:32.112715 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:32.112617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" event={"ID":"bfba5bce-2950-4220-840d-759aeb103c54","Type":"ContainerStarted","Data":"ea114ceafd4325c1e057090e1db1e6dcb5f842b9cf0d56197b03073ed5a08b71"}
Sep 10 16:34:32.132741 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:32.132687    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" podStartSLOduration=1.802519907 podStartE2EDuration="2.132671408s" totalImagesPullingTime="330.151501ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:34:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:34:32.129638667 +0000 UTC m=+1044.531439786" watchObservedRunningTime="2026-09-10 16:34:32.132671408 +0000 UTC m=+1044.534472534"
Sep 10 16:34:38.128530 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:38.128491    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:34:43.800909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:43.800870    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": read tcp 10.132.0.2:55698->10.132.0.77:15021: read: connection reset by peer" start-of-body=
Sep 10 16:34:43.801376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:43.800984    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": read tcp 10.132.0.2:55698->10.132.0.77:15021: read: connection reset by peer"
Sep 10 16:34:44.193392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.193357    2577 generic.go:356] "Generic (PLEG): container finished" podID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerID="55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34" exitCode=0
Sep 10 16:34:44.193573 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.193438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerDied","Data":"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"}
Sep 10 16:34:44.193956 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.193938    2577 scope.go:117] "RemoveContainer" containerID="55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"
Sep 10 16:34:44.195089 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.195058    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:44.195160 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.195145    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:44.195207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:44.195190    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:34:45.202053 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:45.202019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerStarted","Data":"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"}
Sep 10 16:34:46.175896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:46.175848    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:46.176101 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:46.176037    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Sep 10 16:34:46.176177 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:46.176095    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Sep 10 16:34:47.176289 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:47.176248    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Sep 10 16:34:47.176705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:47.176345    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Sep 10 16:34:48.175752 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:48.175715    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Sep 10 16:34:48.175997 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:48.175795    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Sep 10 16:34:49.176197 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:49.176162    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Sep 10 16:34:49.176661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:49.176227    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Sep 10 16:34:50.175713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:50.175680    2577 patch_prober.go:28] interesting pod/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Sep 10 16:34:50.175894 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:50.175737    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Sep 10 16:34:51.179455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:51.179427    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:51.179834 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:51.179758    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:51.180512 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:51.180492    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:34:54.043107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.043073    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:34:54.046428 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.046406    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.049324 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.049301    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-bd1ece-serving-cert\""
Sep 10 16:34:54.059196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.059167    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:34:54.198792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.198752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpqmp\" (UniqueName: \"kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.199096 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.199066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.199241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.199218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.199305 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.199256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.300339 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.300216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpqmp\" (UniqueName: \"kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.300339 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.300307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.300546 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.300367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.300546 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.300386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.301053 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.301030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.302740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.302707    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.302861 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.302798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.312199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.312173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpqmp\" (UniqueName: \"kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp\") pod \"maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.361237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.361188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:54.647957 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:54.647928    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:34:54.648934 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:34:54.648904    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90435514_f49b_4e17_9e2f_a197c6f5b061.slice/crio-9403a43968d8c6913d7e82782dc306842ce26018e68bd12e2940c3928c1c4d34 WatchSource:0}: Error finding container 9403a43968d8c6913d7e82782dc306842ce26018e68bd12e2940c3928c1c4d34: Status 404 returned error can't find the container with id 9403a43968d8c6913d7e82782dc306842ce26018e68bd12e2940c3928c1c4d34
Sep 10 16:34:55.277533 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:55.277440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" event={"ID":"90435514-f49b-4e17-9e2f-a197c6f5b061","Type":"ContainerStarted","Data":"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"}
Sep 10 16:34:55.277533 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:55.277497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" event={"ID":"90435514-f49b-4e17-9e2f-a197c6f5b061","Type":"ContainerStarted","Data":"9403a43968d8c6913d7e82782dc306842ce26018e68bd12e2940c3928c1c4d34"}
Sep 10 16:34:55.277533 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:55.277517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:34:55.295888 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:55.295829    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" podStartSLOduration=0.991487253 podStartE2EDuration="1.295811294s" totalImagesPullingTime="304.324041ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:34:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:34:55.294155895 +0000 UTC m=+1067.695957022" watchObservedRunningTime="2026-09-10 16:34:55.295811294 +0000 UTC m=+1067.697612421"
Sep 10 16:34:58.935880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:58.935842    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:34:58.938803 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:58.938779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:58.941542 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:58.941518    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-3ccbd4-serving-cert\""
Sep 10 16:34:58.952787 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:58.952757    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:34:59.043716 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.043680    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.043958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.043820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.043958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.043940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mswm8\" (UniqueName: \"kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.044096 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.043972    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.144846 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.144802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.145049 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.144893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mswm8\" (UniqueName: \"kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.145049 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.144921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.145049 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.144947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.145607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.145583    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.147398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.147377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.147490 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.147414    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.152778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.152757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mswm8\" (UniqueName: \"kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8\") pod \"maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.253625 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.253543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:34:59.537246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:34:59.537218    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:34:59.538157 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:34:59.538132    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde3b4eec_9782_442c_a68e_0fa9762cb778.slice/crio-30c5d6695fa26e5e3a24505d62bdf3e1984fdad26e15e8074f779161100577a9 WatchSource:0}: Error finding container 30c5d6695fa26e5e3a24505d62bdf3e1984fdad26e15e8074f779161100577a9: Status 404 returned error can't find the container with id 30c5d6695fa26e5e3a24505d62bdf3e1984fdad26e15e8074f779161100577a9
Sep 10 16:35:00.308153 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:00.308042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" event={"ID":"de3b4eec-9782-442c-a68e-0fa9762cb778","Type":"ContainerStarted","Data":"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"}
Sep 10 16:35:00.308153 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:00.308093    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" event={"ID":"de3b4eec-9782-442c-a68e-0fa9762cb778","Type":"ContainerStarted","Data":"30c5d6695fa26e5e3a24505d62bdf3e1984fdad26e15e8074f779161100577a9"}
Sep 10 16:35:00.308600 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:00.308160    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:35:00.327903 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:00.327841    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" podStartSLOduration=1.954926592 podStartE2EDuration="2.327824285s" totalImagesPullingTime="372.897693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:00.325549084 +0000 UTC m=+1072.727350212" watchObservedRunningTime="2026-09-10 16:35:00.327824285 +0000 UTC m=+1072.729625413"
Sep 10 16:35:01.289773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:01.289740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:35:06.319684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:06.319656    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:35:21.253860 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.253798    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:35:21.258954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.258927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.261494 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.261471    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-035bd5-serving-cert\""
Sep 10 16:35:21.271850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.269443    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:35:21.351373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.351340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.351584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.351396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.351584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.351415    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fq2l\" (UniqueName: \"kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.351584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.351524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.452685 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.452651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.452857 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.452716    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.452857 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.452751    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.452857 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.452766    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fq2l\" (UniqueName: \"kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.453039 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:21.452874    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-serving-cert: secret "maas-api-e2e-shared-b-w2-035bd5-serving-cert" not found
Sep 10 16:35:21.453039 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:21.452959    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls podName:0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1 nodeName:}" failed. No retries permitted until 2026-09-10 16:35:21.952939106 +0000 UTC m=+1094.354740213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls") pod "maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" (UID: "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1") : secret "maas-api-e2e-shared-b-w2-035bd5-serving-cert" not found
Sep 10 16:35:21.453344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.453320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.455248 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.455227    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.461874 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.461853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fq2l\" (UniqueName: \"kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.958092 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.958054    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:21.960586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:21.960564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:22.172556 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:22.172526    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:22.450595 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:22.450566    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:35:22.451713 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:22.451688    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fdf9fe6_0b34_4ea5_9c18_b81ed65640a1.slice/crio-11b7c825ed79daa158f12815e5b28b064409e5e13cb5eccde619a913e22bee71 WatchSource:0}: Error finding container 11b7c825ed79daa158f12815e5b28b064409e5e13cb5eccde619a913e22bee71: Status 404 returned error can't find the container with id 11b7c825ed79daa158f12815e5b28b064409e5e13cb5eccde619a913e22bee71
Sep 10 16:35:23.452004 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:23.451967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" event={"ID":"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1","Type":"ContainerStarted","Data":"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"}
Sep 10 16:35:23.452424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:23.452058    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" event={"ID":"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1","Type":"ContainerStarted","Data":"11b7c825ed79daa158f12815e5b28b064409e5e13cb5eccde619a913e22bee71"}
Sep 10 16:35:23.452424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:23.452116    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:23.472023 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:23.471971    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" podStartSLOduration=2.113236005 podStartE2EDuration="2.471955398s" totalImagesPullingTime="358.719393ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:23.469760841 +0000 UTC m=+1095.871561967" watchObservedRunningTime="2026-09-10 16:35:23.471955398 +0000 UTC m=+1095.873756524"
Sep 10 16:35:29.464662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:29.464630    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:35:45.287586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.287550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-77296c"]
Sep 10 16:35:45.291952 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.291932    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:45.295011 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.294982    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mt6ps\""
Sep 10 16:35:45.296872 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.296851    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-77296c"]
Sep 10 16:35:45.388588 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.388557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j4fp\" (UniqueName: \"kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp\") pod \"test-curl-7676-77296c\" (UID: \"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3\") " pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:45.489328 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.489266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j4fp\" (UniqueName: \"kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp\") pod \"test-curl-7676-77296c\" (UID: \"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3\") " pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:45.499313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.499256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j4fp\" (UniqueName: \"kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp\") pod \"test-curl-7676-77296c\" (UID: \"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3\") " pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:45.603631 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.603381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:45.876837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:45.876812    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-77296c"]
Sep 10 16:35:45.881503 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:45.881341    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4c4c3c_dd00_4243_81ab_70a1d8bc29e3.slice/crio-af78882b047070a29f7f3498cb8dd51f1d2b575f0fe6fac46a1dd2ba41c09046 WatchSource:0}: Error finding container af78882b047070a29f7f3498cb8dd51f1d2b575f0fe6fac46a1dd2ba41c09046: Status 404 returned error can't find the container with id af78882b047070a29f7f3498cb8dd51f1d2b575f0fe6fac46a1dd2ba41c09046
Sep 10 16:35:46.592898 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:46.592863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-77296c" event={"ID":"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3","Type":"ContainerStarted","Data":"167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127"}
Sep 10 16:35:46.592898 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:46.592902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-77296c" event={"ID":"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3","Type":"ContainerStarted","Data":"af78882b047070a29f7f3498cb8dd51f1d2b575f0fe6fac46a1dd2ba41c09046"}
Sep 10 16:35:46.612664 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:46.612609    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-77296c" podStartSLOduration=1.61259435 podStartE2EDuration="1.61259435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:46.611016133 +0000 UTC m=+1119.012817270" watchObservedRunningTime="2026-09-10 16:35:46.61259435 +0000 UTC m=+1119.014395504"
Sep 10 16:35:47.473122 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.473087    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-77296c"]
Sep 10 16:35:47.478778 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.478750    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-77296c"]
Sep 10 16:35:47.597885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.597794    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-5895e4"]
Sep 10 16:35:47.598417 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.598294    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" containerName="test-curl-7676-77296c"
Sep 10 16:35:47.598417 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.598309    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" containerName="test-curl-7676-77296c"
Sep 10 16:35:47.598417 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.598401    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" containerName="test-curl-7676-77296c"
Sep 10 16:35:47.599799 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:47.599774    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-77296c" container="test-curl-7676-77296c"
Sep 10 16:35:47.601776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.601760    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:47.612339 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.611464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-5895e4"]
Sep 10 16:35:47.709599 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.709566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s22j\" (UniqueName: \"kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j\") pod \"test-curl-7676-5895e4\" (UID: \"30d614e7-2729-46bb-abab-b7c7b147f3f4\") " pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:47.810055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.810024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s22j\" (UniqueName: \"kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j\") pod \"test-curl-7676-5895e4\" (UID: \"30d614e7-2729-46bb-abab-b7c7b147f3f4\") " pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:47.820113 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.820089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s22j\" (UniqueName: \"kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j\") pod \"test-curl-7676-5895e4\" (UID: \"30d614e7-2729-46bb-abab-b7c7b147f3f4\") " pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:47.912823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:47.912787    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:48.185975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:48.185945    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-5895e4"]
Sep 10 16:35:48.187481 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:48.187452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod30d614e7_2729_46bb_abab_b7c7b147f3f4.slice/crio-c6a75daae93e7e6c6f06756a8db94dd6c6e4aea6a215e64521e16181a618cb44 WatchSource:0}: Error finding container c6a75daae93e7e6c6f06756a8db94dd6c6e4aea6a215e64521e16181a618cb44: Status 404 returned error can't find the container with id c6a75daae93e7e6c6f06756a8db94dd6c6e4aea6a215e64521e16181a618cb44
Sep 10 16:35:48.606475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:48.606388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-5895e4" event={"ID":"30d614e7-2729-46bb-abab-b7c7b147f3f4","Type":"ContainerStarted","Data":"e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727"}
Sep 10 16:35:48.606475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:48.606435    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-5895e4" event={"ID":"30d614e7-2729-46bb-abab-b7c7b147f3f4","Type":"ContainerStarted","Data":"c6a75daae93e7e6c6f06756a8db94dd6c6e4aea6a215e64521e16181a618cb44"}
Sep 10 16:35:48.606990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:48.606518    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-77296c" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" containerName="test-curl-7676-77296c" containerID="cri-o://167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127" gracePeriod=2
Sep 10 16:35:48.626648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:48.626604    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-5895e4" podStartSLOduration=1.626589111 podStartE2EDuration="1.626589111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:48.625626247 +0000 UTC m=+1121.027427374" watchObservedRunningTime="2026-09-10 16:35:48.626589111 +0000 UTC m=+1121.028390285"
Sep 10 16:35:49.277958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.277879    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-5895e4"]
Sep 10 16:35:49.284652 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.284625    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-5895e4"]
Sep 10 16:35:49.394082 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.394052    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-61cb78"]
Sep 10 16:35:49.394577 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.394558    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" containerName="test-curl-7676-5895e4"
Sep 10 16:35:49.394692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.394579    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" containerName="test-curl-7676-5895e4"
Sep 10 16:35:49.394692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.394677    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" containerName="test-curl-7676-5895e4"
Sep 10 16:35:49.398062 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.398043    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:49.403035 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.403010    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-61cb78"]
Sep 10 16:35:49.424714 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.424682    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dg8l\" (UniqueName: \"kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l\") pod \"test-curl-7676-61cb78\" (UID: \"dcd5a602-b2e0-4036-9255-a6c307c67fac\") " pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:49.525636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.525590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dg8l\" (UniqueName: \"kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l\") pod \"test-curl-7676-61cb78\" (UID: \"dcd5a602-b2e0-4036-9255-a6c307c67fac\") " pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:49.535329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.535250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dg8l\" (UniqueName: \"kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l\") pod \"test-curl-7676-61cb78\" (UID: \"dcd5a602-b2e0-4036-9255-a6c307c67fac\") " pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:49.612595 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:49.612561    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-5895e4" container="test-curl-7676-5895e4"
Sep 10 16:35:49.709344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.709314    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:49.979446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:49.979417    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-61cb78"]
Sep 10 16:35:49.980460 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:49.980430    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddcd5a602_b2e0_4036_9255_a6c307c67fac.slice/crio-51fa357604b79c1381b44173adde4fc7c7c072856deedb812a65c171e1c7e429 WatchSource:0}: Error finding container 51fa357604b79c1381b44173adde4fc7c7c072856deedb812a65c171e1c7e429: Status 404 returned error can't find the container with id 51fa357604b79c1381b44173adde4fc7c7c072856deedb812a65c171e1c7e429
Sep 10 16:35:50.620986 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.620946    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-61cb78" event={"ID":"dcd5a602-b2e0-4036-9255-a6c307c67fac","Type":"ContainerStarted","Data":"6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1"}
Sep 10 16:35:50.621362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.620996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-61cb78" event={"ID":"dcd5a602-b2e0-4036-9255-a6c307c67fac","Type":"ContainerStarted","Data":"51fa357604b79c1381b44173adde4fc7c7c072856deedb812a65c171e1c7e429"}
Sep 10 16:35:50.621362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.621197    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-5895e4" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" containerName="test-curl-7676-5895e4" containerID="cri-o://e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727" gracePeriod=2
Sep 10 16:35:50.639850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.639807    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-61cb78" podStartSLOduration=1.639792263 podStartE2EDuration="1.639792263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:50.639193812 +0000 UTC m=+1123.040994940" watchObservedRunningTime="2026-09-10 16:35:50.639792263 +0000 UTC m=+1123.041593389"
Sep 10 16:35:50.797665 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.797645    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:50.836549 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.836524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8j4fp\" (UniqueName: \"kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp\") pod \"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3\" (UID: \"2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3\") "
Sep 10 16:35:50.838875 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.838851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp" (OuterVolumeSpecName: "kube-api-access-8j4fp") pod "2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" (UID: "2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3"). InnerVolumeSpecName "kube-api-access-8j4fp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:35:50.937346 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:50.937313    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8j4fp\" (UniqueName: \"kubernetes.io/projected/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3-kube-api-access-8j4fp\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:35:51.232832 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.232729    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-61cb78"]
Sep 10 16:35:51.237021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.236988    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-61cb78"]
Sep 10 16:35:51.345609 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.345573    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7676-b319a9"]
Sep 10 16:35:51.346018 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.346004    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" containerName="test-curl-7676-61cb78"
Sep 10 16:35:51.346074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.346020    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" containerName="test-curl-7676-61cb78"
Sep 10 16:35:51.346126 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.346105    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" containerName="test-curl-7676-61cb78"
Sep 10 16:35:51.349222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.349201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:51.353971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.353944    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-b319a9"]
Sep 10 16:35:51.440251 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.440215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5h6\" (UniqueName: \"kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6\") pod \"test-curl-7676-b319a9\" (UID: \"1c1df992-58bb-4933-ad29-b0d284b8763d\") " pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:51.540825 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.540736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5h6\" (UniqueName: \"kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6\") pod \"test-curl-7676-b319a9\" (UID: \"1c1df992-58bb-4933-ad29-b0d284b8763d\") " pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:51.551509 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.551475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5h6\" (UniqueName: \"kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6\") pod \"test-curl-7676-b319a9\" (UID: \"1c1df992-58bb-4933-ad29-b0d284b8763d\") " pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:51.628345 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.628308    2577 generic.go:356] "Generic (PLEG): container finished" podID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" containerID="167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127" exitCode=137
Sep 10 16:35:51.628345 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.628335    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:51.628784 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.628401    2577 scope.go:117] "RemoveContainer" containerID="167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127"
Sep 10 16:35:51.628784 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:51.628669    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-61cb78" container="test-curl-7676-61cb78"
Sep 10 16:35:51.638060 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.638042    2577 scope.go:117] "RemoveContainer" containerID="167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127"
Sep 10 16:35:51.638365 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:51.638346    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127\": container with ID starting with 167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127 not found: ID does not exist" containerID="167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127"
Sep 10 16:35:51.638457 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.638373    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127"} err="failed to get container status \"167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127\": rpc error: code = NotFound desc = could not find container \"167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127\": container with ID starting with 167805b9b0d5e141eb8a8ac61316591106744f9c71feab6e3c0948dbf1bbc127 not found: ID does not exist"
Sep 10 16:35:51.639804 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:51.639785    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7676-77296c\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" pod="opendatahub/test-curl-7676-77296c"
Sep 10 16:35:51.660726 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.660697    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:51.944549 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:51.944519    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7676-b319a9"]
Sep 10 16:35:51.945772 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:51.945743    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1c1df992_58bb_4933_ad29_b0d284b8763d.slice/crio-0c38763a21056c93f2370252982dd89fa9f2722c44f750a2e830faef832dcbfa WatchSource:0}: Error finding container 0c38763a21056c93f2370252982dd89fa9f2722c44f750a2e830faef832dcbfa: Status 404 returned error can't find the container with id 0c38763a21056c93f2370252982dd89fa9f2722c44f750a2e830faef832dcbfa
Sep 10 16:35:52.113610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.113577    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3" path="/var/lib/kubelet/pods/2e4c4c3c-dd00-4243-81ab-70a1d8bc29e3/volumes"
Sep 10 16:35:52.637506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.637220    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-b319a9" event={"ID":"1c1df992-58bb-4933-ad29-b0d284b8763d","Type":"ContainerStarted","Data":"0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c"}
Sep 10 16:35:52.637506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.637267    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7676-b319a9" event={"ID":"1c1df992-58bb-4933-ad29-b0d284b8763d","Type":"ContainerStarted","Data":"0c38763a21056c93f2370252982dd89fa9f2722c44f750a2e830faef832dcbfa"}
Sep 10 16:35:52.638577 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.638551    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-61cb78" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" containerName="test-curl-7676-61cb78" containerID="cri-o://6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1" gracePeriod=2
Sep 10 16:35:52.670398 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.670342    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7676-b319a9" podStartSLOduration=1.6703234550000001 podStartE2EDuration="1.670323455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:52.658781633 +0000 UTC m=+1125.060582760" watchObservedRunningTime="2026-09-10 16:35:52.670323455 +0000 UTC m=+1125.072124581"
Sep 10 16:35:52.821669 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.821641    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:52.850196 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.850171    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s22j\" (UniqueName: \"kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j\") pod \"30d614e7-2729-46bb-abab-b7c7b147f3f4\" (UID: \"30d614e7-2729-46bb-abab-b7c7b147f3f4\") "
Sep 10 16:35:52.852401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.852371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j" (OuterVolumeSpecName: "kube-api-access-7s22j") pod "30d614e7-2729-46bb-abab-b7c7b147f3f4" (UID: "30d614e7-2729-46bb-abab-b7c7b147f3f4"). InnerVolumeSpecName "kube-api-access-7s22j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:35:52.950973 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:52.950898    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s22j\" (UniqueName: \"kubernetes.io/projected/30d614e7-2729-46bb-abab-b7c7b147f3f4-kube-api-access-7s22j\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:35:53.268006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.267919    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7676-b319a9"]
Sep 10 16:35:53.273053 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.273020    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7676-b319a9"]
Sep 10 16:35:53.646043 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.646003    2577 generic.go:356] "Generic (PLEG): container finished" podID="30d614e7-2729-46bb-abab-b7c7b147f3f4" containerID="e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727" exitCode=137
Sep 10 16:35:53.646554 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.646054    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:53.646554 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.646108    2577 scope.go:117] "RemoveContainer" containerID="e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727"
Sep 10 16:35:53.646554 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:53.646427    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7676-b319a9" container="test-curl-7676-b319a9"
Sep 10 16:35:53.656034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.656014    2577 scope.go:117] "RemoveContainer" containerID="e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727"
Sep 10 16:35:53.656420 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:53.656394    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727\": container with ID starting with e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727 not found: ID does not exist" containerID="e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727"
Sep 10 16:35:53.656495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:53.656427    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727"} err="failed to get container status \"e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727\": rpc error: code = NotFound desc = could not find container \"e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727\": container with ID starting with e4214f9cda2ff64919f656efd8221d90ed8316e2abd76182361e555bb3a8d727 not found: ID does not exist"
Sep 10 16:35:53.657688 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:53.657665    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7676-5895e4\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" pod="opendatahub/test-curl-7676-5895e4"
Sep 10 16:35:54.113617 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.113589    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30d614e7-2729-46bb-abab-b7c7b147f3f4" path="/var/lib/kubelet/pods/30d614e7-2729-46bb-abab-b7c7b147f3f4/volumes"
Sep 10 16:35:54.198457 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.198424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:35:54.198876 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.198864    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" containerName="test-curl-7676-b319a9"
Sep 10 16:35:54.198954 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.198878    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" containerName="test-curl-7676-b319a9"
Sep 10 16:35:54.199008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.198954    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" containerName="test-curl-7676-b319a9"
Sep 10 16:35:54.203694 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.203675    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.207315 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.207294    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8438b933-openshift-default-dockercfg-m8xsc\""
Sep 10 16:35:54.218943 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.218921    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:35:54.264264 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264231    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264264 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264346    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wg7\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264511 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264508    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.264707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.264586    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.365985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366037    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366108    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366134    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366162    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96wg7\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366206    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366313 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366645 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366645 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366437    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366645 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366699    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.366799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.366788    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.367086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.367064    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.368626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.368606    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.368816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.368798    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.376335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.376315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.376537 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.376516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wg7\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7\") pod \"e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") " pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.515371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.515335    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:54.653221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.653158    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7676-b319a9" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" containerName="test-curl-7676-b319a9" containerID="cri-o://0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c" gracePeriod=2
Sep 10 16:35:54.803688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.803658    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:35:54.804743 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:35:54.804709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb9d6dd_4169_485e_8a46_c995591a6a09.slice/crio-80b02997e073bd4a972e2c8882e579485cad643b71d475431a1f71217cc74b25 WatchSource:0}: Error finding container 80b02997e073bd4a972e2c8882e579485cad643b71d475431a1f71217cc74b25: Status 404 returned error can't find the container with id 80b02997e073bd4a972e2c8882e579485cad643b71d475431a1f71217cc74b25
Sep 10 16:35:54.806948 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.806914    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:35:54.807068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.806990    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:35:54.807068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.807035    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:35:54.826813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.826787    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:54.871022 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.871000    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dg8l\" (UniqueName: \"kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l\") pod \"dcd5a602-b2e0-4036-9255-a6c307c67fac\" (UID: \"dcd5a602-b2e0-4036-9255-a6c307c67fac\") "
Sep 10 16:35:54.873049 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.873025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l" (OuterVolumeSpecName: "kube-api-access-2dg8l") pod "dcd5a602-b2e0-4036-9255-a6c307c67fac" (UID: "dcd5a602-b2e0-4036-9255-a6c307c67fac"). InnerVolumeSpecName "kube-api-access-2dg8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:35:54.973029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:54.972941    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dg8l\" (UniqueName: \"kubernetes.io/projected/dcd5a602-b2e0-4036-9255-a6c307c67fac-kube-api-access-2dg8l\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:35:55.660718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.660627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerStarted","Data":"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"}
Sep 10 16:35:55.660718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.660687    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerStarted","Data":"80b02997e073bd4a972e2c8882e579485cad643b71d475431a1f71217cc74b25"}
Sep 10 16:35:55.661951 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.661928    2577 generic.go:356] "Generic (PLEG): container finished" podID="dcd5a602-b2e0-4036-9255-a6c307c67fac" containerID="6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1" exitCode=137
Sep 10 16:35:55.662021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.661962    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:55.662056 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.662033    2577 scope.go:117] "RemoveContainer" containerID="6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1"
Sep 10 16:35:55.671905 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.671887    2577 scope.go:117] "RemoveContainer" containerID="6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1"
Sep 10 16:35:55.672174 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:55.672157    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1\": container with ID starting with 6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1 not found: ID does not exist" containerID="6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1"
Sep 10 16:35:55.672217 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.672183    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1"} err="failed to get container status \"6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1\": rpc error: code = NotFound desc = could not find container \"6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1\": container with ID starting with 6e02cf2c505e4c890f0bb884cab5aeefb70933b9e3ac9196a3d085fbc63186d1 not found: ID does not exist"
Sep 10 16:35:55.690073 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:55.690033    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podStartSLOduration=1.690018739 podStartE2EDuration="1.690018739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:35:55.689766417 +0000 UTC m=+1128.091567544" watchObservedRunningTime="2026-09-10 16:35:55.690018739 +0000 UTC m=+1128.091819866"
Sep 10 16:35:55.692545 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:55.692518    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7676-61cb78\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" pod="opendatahub/test-curl-7676-61cb78"
Sep 10 16:35:56.114242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.114214    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dcd5a602-b2e0-4036-9255-a6c307c67fac" path="/var/lib/kubelet/pods/dcd5a602-b2e0-4036-9255-a6c307c67fac/volumes"
Sep 10 16:35:56.516146 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.516061    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:56.521421 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.521394    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:56.669950 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.669928    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:56.671034 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.671017    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:35:56.851667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.851638    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:56.891975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.891938    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk5h6\" (UniqueName: \"kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6\") pod \"1c1df992-58bb-4933-ad29-b0d284b8763d\" (UID: \"1c1df992-58bb-4933-ad29-b0d284b8763d\") "
Sep 10 16:35:56.894206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.894168    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6" (OuterVolumeSpecName: "kube-api-access-rk5h6") pod "1c1df992-58bb-4933-ad29-b0d284b8763d" (UID: "1c1df992-58bb-4933-ad29-b0d284b8763d"). InnerVolumeSpecName "kube-api-access-rk5h6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:35:56.992764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:56.992723    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk5h6\" (UniqueName: \"kubernetes.io/projected/1c1df992-58bb-4933-ad29-b0d284b8763d-kube-api-access-rk5h6\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:35:57.689066 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:57.688924    2577 generic.go:356] "Generic (PLEG): container finished" podID="1c1df992-58bb-4933-ad29-b0d284b8763d" containerID="0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c" exitCode=137
Sep 10 16:35:57.689066 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:57.689002    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:57.689610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:57.689120    2577 scope.go:117] "RemoveContainer" containerID="0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c"
Sep 10 16:35:57.701021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:57.700797    2577 scope.go:117] "RemoveContainer" containerID="0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c"
Sep 10 16:35:57.701135 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:57.701114    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c\": container with ID starting with 0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c not found: ID does not exist" containerID="0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c"
Sep 10 16:35:57.701175 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:57.701141    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c"} err="failed to get container status \"0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c\": rpc error: code = NotFound desc = could not find container \"0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c\": container with ID starting with 0a595a8b367c304ac047508feee5dcbbe9908ee6c3c20ad56fe9638a38bc820c not found: ID does not exist"
Sep 10 16:35:57.702886 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:57.702863    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7676-b319a9\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:58.112486 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:35:58.112396    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7676-b319a9\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" pod="opendatahub/test-curl-7676-b319a9"
Sep 10 16:35:58.114210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:35:58.114183    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c1df992-58bb-4933-ad29-b0d284b8763d" path="/var/lib/kubelet/pods/1c1df992-58bb-4933-ad29-b0d284b8763d/volumes"
Sep 10 16:36:00.676345 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.676310    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:36:00.681029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.681009    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.684027 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.684003    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8438b933-serving-cert\""
Sep 10 16:36:00.693679 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.693651    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:36:00.730217 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.730187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.730361 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.730236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz2cf\" (UniqueName: \"kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.730361 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.730334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.730458 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.730428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.831937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.831903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.832110 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.831954    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.832110 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.831992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz2cf\" (UniqueName: \"kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.832110 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.832029    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.832678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.832644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.834419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.834393    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.834518 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.834447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.841524 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.841505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz2cf\" (UniqueName: \"kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf\") pod \"maas-api-e2e-mt-8438b933-bd7c86d95-jzh94\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:00.993656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:00.993564    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:01.275067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:01.275035    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:36:01.276233 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:36:01.276196    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03932c7_ee49_4dbb_9667_312d3628dc49.slice/crio-d162fc588063e66bbb45504d667502d11ffab4596fc7edc4d363303d2096b5a2 WatchSource:0}: Error finding container d162fc588063e66bbb45504d667502d11ffab4596fc7edc4d363303d2096b5a2: Status 404 returned error can't find the container with id d162fc588063e66bbb45504d667502d11ffab4596fc7edc4d363303d2096b5a2
Sep 10 16:36:01.719414 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:01.719378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" event={"ID":"b03932c7-ee49-4dbb-9667-312d3628dc49","Type":"ContainerStarted","Data":"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"}
Sep 10 16:36:01.719776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:01.719436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" event={"ID":"b03932c7-ee49-4dbb-9667-312d3628dc49","Type":"ContainerStarted","Data":"d162fc588063e66bbb45504d667502d11ffab4596fc7edc4d363303d2096b5a2"}
Sep 10 16:36:01.719776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:01.719466    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:01.742648 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:01.742566    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" podStartSLOduration=1.384270198 podStartE2EDuration="1.742545945s" totalImagesPullingTime="358.275747ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:36:01.740836196 +0000 UTC m=+1134.142637323" watchObservedRunningTime="2026-09-10 16:36:01.742545945 +0000 UTC m=+1134.144347071"
Sep 10 16:36:07.731691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:07.731659    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:36:18.833653 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.833617    2577 generic.go:356] "Generic (PLEG): container finished" podID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerID="26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc" exitCode=0
Sep 10 16:36:18.834178 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.833698    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerDied","Data":"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"}
Sep 10 16:36:18.834271 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.834252    2577 scope.go:117] "RemoveContainer" containerID="26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"
Sep 10 16:36:18.835433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.835405    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:36:18.835508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.835472    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:36:18.835547 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:18.835505    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:36:19.841785 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:19.841750    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerStarted","Data":"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"}
Sep 10 16:36:20.516136 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:20.516092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:36:20.516438 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:20.516412    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 10 16:36:20.516541 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:20.516470    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 10 16:36:21.516172 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:21.516129    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 10 16:36:21.516710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:21.516377    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 10 16:36:22.516523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:22.516486    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 10 16:36:22.516984 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:22.516547    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 10 16:36:23.516186 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:23.516142    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 10 16:36:23.516410 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:23.516218    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 10 16:36:24.516040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:24.516000    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 10 16:36:24.516454 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:24.516054    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 10 16:36:25.520549 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:25.520517    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:36:25.521051 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:25.521029    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:36:25.521819 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:36:25.521797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:37:08.152372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:08.152344    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:37:08.154831 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:08.154805    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:37:22.453170 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.453137    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:37:22.453556 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.453391    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" podUID="b03932c7-ee49-4dbb-9667-312d3628dc49" containerName="maas-api" containerID="cri-o://c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691" gracePeriod=30
Sep 10 16:37:22.714839 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.714760    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:37:22.747624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.747590    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca\") pod \"b03932c7-ee49-4dbb-9667-312d3628dc49\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") "
Sep 10 16:37:22.747777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.747735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls\") pod \"b03932c7-ee49-4dbb-9667-312d3628dc49\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") "
Sep 10 16:37:22.747777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.747769    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs\") pod \"b03932c7-ee49-4dbb-9667-312d3628dc49\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") "
Sep 10 16:37:22.747899 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.747796    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qz2cf\" (UniqueName: \"kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf\") pod \"b03932c7-ee49-4dbb-9667-312d3628dc49\" (UID: \"b03932c7-ee49-4dbb-9667-312d3628dc49\") "
Sep 10 16:37:22.747988 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.747959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b03932c7-ee49-4dbb-9667-312d3628dc49" (UID: "b03932c7-ee49-4dbb-9667-312d3628dc49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:37:22.748218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.748195    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b03932c7-ee49-4dbb-9667-312d3628dc49-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:37:22.749898 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.749865    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b03932c7-ee49-4dbb-9667-312d3628dc49" (UID: "b03932c7-ee49-4dbb-9667-312d3628dc49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:37:22.750373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.750349    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf" (OuterVolumeSpecName: "kube-api-access-qz2cf") pod "b03932c7-ee49-4dbb-9667-312d3628dc49" (UID: "b03932c7-ee49-4dbb-9667-312d3628dc49"). InnerVolumeSpecName "kube-api-access-qz2cf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:37:22.750373 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.750355    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b03932c7-ee49-4dbb-9667-312d3628dc49" (UID: "b03932c7-ee49-4dbb-9667-312d3628dc49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:37:22.849469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.849439    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:37:22.849469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.849463    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b03932c7-ee49-4dbb-9667-312d3628dc49-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:37:22.849469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:22.849472    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz2cf\" (UniqueName: \"kubernetes.io/projected/b03932c7-ee49-4dbb-9667-312d3628dc49-kube-api-access-qz2cf\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:37:23.236709 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.236670    2577 generic.go:356] "Generic (PLEG): container finished" podID="b03932c7-ee49-4dbb-9667-312d3628dc49" containerID="c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691" exitCode=0
Sep 10 16:37:23.236892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.236720    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"
Sep 10 16:37:23.236892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.236755    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" event={"ID":"b03932c7-ee49-4dbb-9667-312d3628dc49","Type":"ContainerDied","Data":"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"}
Sep 10 16:37:23.236892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.236818    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94" event={"ID":"b03932c7-ee49-4dbb-9667-312d3628dc49","Type":"ContainerDied","Data":"d162fc588063e66bbb45504d667502d11ffab4596fc7edc4d363303d2096b5a2"}
Sep 10 16:37:23.236892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.236841    2577 scope.go:117] "RemoveContainer" containerID="c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"
Sep 10 16:37:23.247256 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.247239    2577 scope.go:117] "RemoveContainer" containerID="c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"
Sep 10 16:37:23.247550 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:37:23.247528    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691\": container with ID starting with c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691 not found: ID does not exist" containerID="c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"
Sep 10 16:37:23.247596 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.247558    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691"} err="failed to get container status \"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691\": rpc error: code = NotFound desc = could not find container \"c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691\": container with ID starting with c728db2dedd83aea47dd68810770b221ea5ecb6e4339a9c60f480e195c1b5691 not found: ID does not exist"
Sep 10 16:37:23.262446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.262423    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:37:23.270333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:23.270310    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8438b933-bd7c86d95-jzh94"]
Sep 10 16:37:24.115176 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:37:24.115140    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b03932c7-ee49-4dbb-9667-312d3628dc49" path="/var/lib/kubelet/pods/b03932c7-ee49-4dbb-9667-312d3628dc49/volumes"
Sep 10 16:38:29.259769 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.259736    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7679-4cdc45"]
Sep 10 16:38:29.260242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.260204    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b03932c7-ee49-4dbb-9667-312d3628dc49" containerName="maas-api"
Sep 10 16:38:29.260242 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.260216    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b03932c7-ee49-4dbb-9667-312d3628dc49" containerName="maas-api"
Sep 10 16:38:29.260339 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.260301    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b03932c7-ee49-4dbb-9667-312d3628dc49" containerName="maas-api"
Sep 10 16:38:29.264733 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.264715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:29.268332 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.268304    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mt6ps\""
Sep 10 16:38:29.271361 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.271322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-4cdc45"]
Sep 10 16:38:29.408696 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.408659    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x479f\" (UniqueName: \"kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f\") pod \"test-curl-7679-4cdc45\" (UID: \"e11bba7e-e624-473a-a769-463c90f82140\") " pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:29.509987 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.509893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x479f\" (UniqueName: \"kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f\") pod \"test-curl-7679-4cdc45\" (UID: \"e11bba7e-e624-473a-a769-463c90f82140\") " pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:29.519150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.519117    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x479f\" (UniqueName: \"kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f\") pod \"test-curl-7679-4cdc45\" (UID: \"e11bba7e-e624-473a-a769-463c90f82140\") " pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:29.575142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.575103    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:29.859419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:29.855435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-4cdc45"]
Sep 10 16:38:29.861269 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:38:29.861235    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode11bba7e_e624_473a_a769_463c90f82140.slice/crio-d8696b897db47812b65f87ae4e4394323ade2087443fd56328f7e763eac07e28 WatchSource:0}: Error finding container d8696b897db47812b65f87ae4e4394323ade2087443fd56328f7e763eac07e28: Status 404 returned error can't find the container with id d8696b897db47812b65f87ae4e4394323ade2087443fd56328f7e763eac07e28
Sep 10 16:38:30.665164 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:30.665124    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-4cdc45" event={"ID":"e11bba7e-e624-473a-a769-463c90f82140","Type":"ContainerStarted","Data":"4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e"}
Sep 10 16:38:30.665564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:30.665177    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-4cdc45" event={"ID":"e11bba7e-e624-473a-a769-463c90f82140","Type":"ContainerStarted","Data":"d8696b897db47812b65f87ae4e4394323ade2087443fd56328f7e763eac07e28"}
Sep 10 16:38:30.684448 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:30.684399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7679-4cdc45" podStartSLOduration=1.684384383 podStartE2EDuration="1.684384383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:38:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:38:30.682736422 +0000 UTC m=+1283.084537560" watchObservedRunningTime="2026-09-10 16:38:30.684384383 +0000 UTC m=+1283.086185509"
Sep 10 16:38:31.266496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:31.266458    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7679-4cdc45"]
Sep 10 16:38:31.275752 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:31.275722    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7679-4cdc45"]
Sep 10 16:38:31.671433 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:38:31.671398    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7679-4cdc45" container="test-curl-7679-4cdc45"
Sep 10 16:38:32.676960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:32.676920    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7679-4cdc45" podUID="e11bba7e-e624-473a-a769-463c90f82140" containerName="test-curl-7679-4cdc45" containerID="cri-o://4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e" gracePeriod=2
Sep 10 16:38:34.854296 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:34.854247    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:34.966651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:34.966554    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x479f\" (UniqueName: \"kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f\") pod \"e11bba7e-e624-473a-a769-463c90f82140\" (UID: \"e11bba7e-e624-473a-a769-463c90f82140\") "
Sep 10 16:38:34.968658 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:34.968630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f" (OuterVolumeSpecName: "kube-api-access-x479f") pod "e11bba7e-e624-473a-a769-463c90f82140" (UID: "e11bba7e-e624-473a-a769-463c90f82140"). InnerVolumeSpecName "kube-api-access-x479f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:38:35.067589 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.067545    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x479f\" (UniqueName: \"kubernetes.io/projected/e11bba7e-e624-473a-a769-463c90f82140-kube-api-access-x479f\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:35.698306 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.698250    2577 generic.go:356] "Generic (PLEG): container finished" podID="e11bba7e-e624-473a-a769-463c90f82140" containerID="4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e" exitCode=137
Sep 10 16:38:35.698517 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.698320    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:35.698517 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.698353    2577 scope.go:117] "RemoveContainer" containerID="4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e"
Sep 10 16:38:35.708536 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.708516    2577 scope.go:117] "RemoveContainer" containerID="4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e"
Sep 10 16:38:35.708873 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:38:35.708843    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e\": container with ID starting with 4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e not found: ID does not exist" containerID="4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e"
Sep 10 16:38:35.708982 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:35.708884    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e"} err="failed to get container status \"4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e\": rpc error: code = NotFound desc = could not find container \"4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e\": container with ID starting with 4e016d33c90fb6b3a80ca098b8ddeb6ab6cc72b1c346fcdf632e48c24617454e not found: ID does not exist"
Sep 10 16:38:35.709034 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:38:35.708978    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7679-4cdc45\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="e11bba7e-e624-473a-a769-463c90f82140" pod="opendatahub/test-curl-7679-4cdc45"
Sep 10 16:38:36.113000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:36.112973    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e11bba7e-e624-473a-a769-463c90f82140" path="/var/lib/kubelet/pods/e11bba7e-e624-473a-a769-463c90f82140/volumes"
Sep 10 16:38:38.149450 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:38.149417    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:38:38.149848 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:38.149614    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" containerID="cri-o://d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540" gracePeriod=30
Sep 10 16:38:40.523026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:40.522974    2577 patch_prober.go:28] interesting pod/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 16:38:40.523446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:40.523054    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 16:38:43.415064 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.414990    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:38:43.449527 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449490    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449550    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449596    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449637    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96wg7\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449668    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449785    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449806    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data" (OuterVolumeSpecName: "istio-data") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:38:43.449990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy\") pod \"efb9d6dd-4169-485e-8a46-c995591a6a09\" (UID: \"efb9d6dd-4169-485e-8a46-c995591a6a09\") "
Sep 10 16:38:43.449990 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.449859    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:38:43.450244 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450061    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:38:43.450331 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450301    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:38:43.450396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450357    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-data\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.450396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450376    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.450396 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450390    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-credential-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.450529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.450386    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:38:43.451989 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.451961    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7" (OuterVolumeSpecName: "kube-api-access-96wg7") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "kube-api-access-96wg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:38:43.452266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.452251    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:38:43.452436 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.452370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token" (OuterVolumeSpecName: "istio-token") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:38:43.452436 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.452395    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "efb9d6dd-4169-485e-8a46-c995591a6a09" (UID: "efb9d6dd-4169-485e-8a46-c995591a6a09"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 16:38:43.551492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551451    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-podinfo\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.551492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551486    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96wg7\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-kube-api-access-96wg7\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.551492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551496    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-workload-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.551492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551505    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/efb9d6dd-4169-485e-8a46-c995591a6a09-istiod-ca-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.551774 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551513    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.551774 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.551521    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/efb9d6dd-4169-485e-8a46-c995591a6a09-istio-envoy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:38:43.754586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.754495    2577 generic.go:356] "Generic (PLEG): container finished" podID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerID="d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540" exitCode=0
Sep 10 16:38:43.754586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.754552    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"
Sep 10 16:38:43.754764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.754551    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerDied","Data":"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"}
Sep 10 16:38:43.754764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.754671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw" event={"ID":"efb9d6dd-4169-485e-8a46-c995591a6a09","Type":"ContainerDied","Data":"80b02997e073bd4a972e2c8882e579485cad643b71d475431a1f71217cc74b25"}
Sep 10 16:38:43.754764 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.754702    2577 scope.go:117] "RemoveContainer" containerID="d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"
Sep 10 16:38:43.765166 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.765148    2577 scope.go:117] "RemoveContainer" containerID="26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"
Sep 10 16:38:43.774430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.774332    2577 scope.go:117] "RemoveContainer" containerID="d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"
Sep 10 16:38:43.774970 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:38:43.774818    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540\": container with ID starting with d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540 not found: ID does not exist" containerID="d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"
Sep 10 16:38:43.774970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.774868    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540"} err="failed to get container status \"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540\": rpc error: code = NotFound desc = could not find container \"d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540\": container with ID starting with d3ddb17480fc47e4eebfbb1dbce557cea1b4988936aa3458fba316608efdf540 not found: ID does not exist"
Sep 10 16:38:43.774970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.774890    2577 scope.go:117] "RemoveContainer" containerID="26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"
Sep 10 16:38:43.775678 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:38:43.775184    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc\": container with ID starting with 26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc not found: ID does not exist" containerID="26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"
Sep 10 16:38:43.775678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.775210    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc"} err="failed to get container status \"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc\": rpc error: code = NotFound desc = could not find container \"26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc\": container with ID starting with 26542e978fc15e4548fc93ade01931b5a551d1f793f3f04c918d5b49470067dc not found: ID does not exist"
Sep 10 16:38:43.778188 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.778165    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:38:43.782145 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:43.782122    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8438b933-openshift-default-7fcc46cb-pjscw"]
Sep 10 16:38:44.114148 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:38:44.114121    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" path="/var/lib/kubelet/pods/efb9d6dd-4169-485e-8a46-c995591a6a09/volumes"
Sep 10 16:39:34.050382 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050301    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:39:34.050924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050890    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e11bba7e-e624-473a-a769-463c90f82140" containerName="test-curl-7679-4cdc45"
Sep 10 16:39:34.050924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050916    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e11bba7e-e624-473a-a769-463c90f82140" containerName="test-curl-7679-4cdc45"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050959    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050968    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050979    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.050988    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.051068    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:34.051123 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.051078    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e11bba7e-e624-473a-a769-463c90f82140" containerName="test-curl-7679-4cdc45"
Sep 10 16:39:34.062310 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.062259    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.065515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.065490    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-fb3d69-openshift-default-dockercfg-hxx8h\""
Sep 10 16:39:34.072918 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.072892    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:39:34.140576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140544    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.140815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140598    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.140815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.140815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.140815 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140754    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw2js\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.141068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140836    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.141068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.141068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140970    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.141068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.140993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.241718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.241718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241763    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.241884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw2js\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242086    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242642 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242597    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.242688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.242673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.244454 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.244431    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.244564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.244455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.254169 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.254140    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.254462 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.254441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw2js\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js\") pod \"e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") " pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.375070 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.375031    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:34.666237 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.666212    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:39:34.668042 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:39:34.668014    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3db76c33_1c5c_4c8c_92a0_812c1aa2cb00.slice/crio-e3d80fbf0961c12e44a1e3eedcd0b0f1734ea9ce27d909987c29cf0a03466adb WatchSource:0}: Error finding container e3d80fbf0961c12e44a1e3eedcd0b0f1734ea9ce27d909987c29cf0a03466adb: Status 404 returned error can't find the container with id e3d80fbf0961c12e44a1e3eedcd0b0f1734ea9ce27d909987c29cf0a03466adb
Sep 10 16:39:34.670244 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.670211    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:39:34.670333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.670300    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:39:34.670375 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:34.670343    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:39:35.073329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:35.073225    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerStarted","Data":"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"}
Sep 10 16:39:35.073329 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:35.073302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerStarted","Data":"e3d80fbf0961c12e44a1e3eedcd0b0f1734ea9ce27d909987c29cf0a03466adb"}
Sep 10 16:39:35.376115 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:35.376078    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:35.381836 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:35.381806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:35.403118 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:35.403063    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podStartSLOduration=1.4030460599999999 podStartE2EDuration="1.40304606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:39:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:39:35.104543331 +0000 UTC m=+1347.506344459" watchObservedRunningTime="2026-09-10 16:39:35.40304606 +0000 UTC m=+1347.804847187"
Sep 10 16:39:36.079767 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:36.079739    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:36.080684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:36.080665    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:39:40.354391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.354357    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:39:40.355000 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.354979    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb9d6dd-4169-485e-8a46-c995591a6a09" containerName="istio-proxy"
Sep 10 16:39:40.358370 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.358343    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.361352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.361320    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-fb3d69-serving-cert\""
Sep 10 16:39:40.376508 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.376479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:39:40.404371 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.404325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.404535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.404378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.404535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.404481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.404535 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.404525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zg97\" (UniqueName: \"kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.506021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.505987    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.506021 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.506024    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.506300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.506073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.506300 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.506118    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zg97\" (UniqueName: \"kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.506300 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:40.506183    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-serving-cert: secret "maas-api-e2e-api-a-fb3d69-serving-cert" not found
Sep 10 16:39:40.506300 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:40.506255    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls podName:cde1201f-e00d-4729-9c29-f9a370826d71 nodeName:}" failed. No retries permitted until 2026-09-10 16:39:41.006234183 +0000 UTC m=+1353.408035301 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls") pod "maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" (UID: "cde1201f-e00d-4729-9c29-f9a370826d71") : secret "maas-api-e2e-api-a-fb3d69-serving-cert" not found
Sep 10 16:39:40.506871 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.506845    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.508621 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.508598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:40.518740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:40.518713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zg97\" (UniqueName: \"kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:41.011824 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:41.011786    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:41.014190 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:41.014159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") pod \"maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:41.270605 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:41.270512    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:41.568322 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:41.568287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:39:41.569173 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:39:41.569147    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde1201f_e00d_4729_9c29_f9a370826d71.slice/crio-c26c34351b781112cd744c0f3a75580471d681f55a762b78f924131de189edae WatchSource:0}: Error finding container c26c34351b781112cd744c0f3a75580471d681f55a762b78f924131de189edae: Status 404 returned error can't find the container with id c26c34351b781112cd744c0f3a75580471d681f55a762b78f924131de189edae
Sep 10 16:39:41.570681 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:41.570663    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:39:42.117937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:42.117904    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" event={"ID":"cde1201f-e00d-4729-9c29-f9a370826d71","Type":"ContainerStarted","Data":"c26c34351b781112cd744c0f3a75580471d681f55a762b78f924131de189edae"}
Sep 10 16:39:43.133088 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:43.133050    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" event={"ID":"cde1201f-e00d-4729-9c29-f9a370826d71","Type":"ContainerStarted","Data":"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"}
Sep 10 16:39:43.133464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:43.133135    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:43.155842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:43.155798    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" podStartSLOduration=2.61145235 podStartE2EDuration="3.155783264s" totalImagesPullingTime="544.330914ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:39:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:39:43.154452843 +0000 UTC m=+1355.556253971" watchObservedRunningTime="2026-09-10 16:39:43.155783264 +0000 UTC m=+1355.557584429"
Sep 10 16:39:49.143777 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:49.143749    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:39:52.747592 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.747553    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7679-031ef0"]
Sep 10 16:39:52.751526 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.751507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:52.754563 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.754542    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mt6ps\""
Sep 10 16:39:52.758602 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.758568    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-031ef0"]
Sep 10 16:39:52.823926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.823879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5czt\" (UniqueName: \"kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt\") pod \"test-curl-7679-031ef0\" (UID: \"6b634c35-f1c5-4433-9345-d1e32f8f0f69\") " pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:52.924753 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.924707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5czt\" (UniqueName: \"kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt\") pod \"test-curl-7679-031ef0\" (UID: \"6b634c35-f1c5-4433-9345-d1e32f8f0f69\") " pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:52.933615 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:52.933580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5czt\" (UniqueName: \"kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt\") pod \"test-curl-7679-031ef0\" (UID: \"6b634c35-f1c5-4433-9345-d1e32f8f0f69\") " pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:53.062129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:53.062038    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:53.373139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:53.373114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-031ef0"]
Sep 10 16:39:53.374966 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:39:53.374938    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6b634c35_f1c5_4433_9345_d1e32f8f0f69.slice/crio-92ff9ad71115c664bd3dfa71dcbc62f2253f4d2841465ba92cd86efecd59122a WatchSource:0}: Error finding container 92ff9ad71115c664bd3dfa71dcbc62f2253f4d2841465ba92cd86efecd59122a: Status 404 returned error can't find the container with id 92ff9ad71115c664bd3dfa71dcbc62f2253f4d2841465ba92cd86efecd59122a
Sep 10 16:39:54.225991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.225954    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-031ef0" event={"ID":"6b634c35-f1c5-4433-9345-d1e32f8f0f69","Type":"ContainerStarted","Data":"47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24"}
Sep 10 16:39:54.225991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.225996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-031ef0" event={"ID":"6b634c35-f1c5-4433-9345-d1e32f8f0f69","Type":"ContainerStarted","Data":"92ff9ad71115c664bd3dfa71dcbc62f2253f4d2841465ba92cd86efecd59122a"}
Sep 10 16:39:54.247026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.246975    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7679-031ef0" podStartSLOduration=2.246959187 podStartE2EDuration="2.246959187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:39:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:39:54.243023601 +0000 UTC m=+1366.644824729" watchObservedRunningTime="2026-09-10 16:39:54.246959187 +0000 UTC m=+1366.648760355"
Sep 10 16:39:54.853887 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.853850    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7679-031ef0"]
Sep 10 16:39:54.858078 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.858053    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7679-031ef0"]
Sep 10 16:39:54.980261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.976570    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7679-09ce27"]
Sep 10 16:39:54.980261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.977775    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" containerName="test-curl-7679-031ef0"
Sep 10 16:39:54.980261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.977798    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" containerName="test-curl-7679-031ef0"
Sep 10 16:39:54.980261 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.978032    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" containerName="test-curl-7679-031ef0"
Sep 10 16:39:54.994372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:54.982509    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:39:55.017470 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.017435    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-09ce27"]
Sep 10 16:39:55.147471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.147435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rdzz\" (UniqueName: \"kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz\") pod \"test-curl-7679-09ce27\" (UID: \"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6\") " pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:39:55.232662 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:55.232582    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7679-031ef0" container="test-curl-7679-031ef0"
Sep 10 16:39:55.248464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.248435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rdzz\" (UniqueName: \"kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz\") pod \"test-curl-7679-09ce27\" (UID: \"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6\") " pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:39:55.256397 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.256373    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rdzz\" (UniqueName: \"kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz\") pod \"test-curl-7679-09ce27\" (UID: \"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6\") " pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:39:55.322341 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.322305    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:39:55.604548 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:55.604453    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7679-09ce27"]
Sep 10 16:39:56.239970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.239928    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-09ce27" event={"ID":"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6","Type":"ContainerStarted","Data":"707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f"}
Sep 10 16:39:56.239970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.239972    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7679-09ce27" event={"ID":"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6","Type":"ContainerStarted","Data":"5a6676729fd50e43b464c0f293ee07c8d11ab5f235a628786aeca787bbf5ed56"}
Sep 10 16:39:56.240471 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.240017    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7679-031ef0" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" containerName="test-curl-7679-031ef0" containerID="cri-o://47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24" gracePeriod=2
Sep 10 16:39:56.262295 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.262238    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7679-09ce27" podStartSLOduration=2.262224848 podStartE2EDuration="2.262224848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:39:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:39:56.259031109 +0000 UTC m=+1368.660832236" watchObservedRunningTime="2026-09-10 16:39:56.262224848 +0000 UTC m=+1368.664025998"
Sep 10 16:39:56.897065 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.897022    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7679-09ce27"]
Sep 10 16:39:56.915410 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:56.914872    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7679-09ce27"]
Sep 10 16:39:57.247695 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:57.247664    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7679-09ce27" container="test-curl-7679-09ce27"
Sep 10 16:39:58.254349 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:58.254287    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7679-09ce27" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" containerName="test-curl-7679-09ce27" containerID="cri-o://707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f" gracePeriod=2
Sep 10 16:39:58.435409 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:58.435381    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:58.578823 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:58.578730    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n5czt\" (UniqueName: \"kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt\") pod \"6b634c35-f1c5-4433-9345-d1e32f8f0f69\" (UID: \"6b634c35-f1c5-4433-9345-d1e32f8f0f69\") "
Sep 10 16:39:58.580896 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:58.580866    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt" (OuterVolumeSpecName: "kube-api-access-n5czt") pod "6b634c35-f1c5-4433-9345-d1e32f8f0f69" (UID: "6b634c35-f1c5-4433-9345-d1e32f8f0f69"). InnerVolumeSpecName "kube-api-access-n5czt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:39:58.680661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:58.680627    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n5czt\" (UniqueName: \"kubernetes.io/projected/6b634c35-f1c5-4433-9345-d1e32f8f0f69-kube-api-access-n5czt\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:39:59.261869 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:59.261833    2577 generic.go:356] "Generic (PLEG): container finished" podID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" containerID="47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24" exitCode=137
Sep 10 16:39:59.262316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:59.261879    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:39:59.262316 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:59.261939    2577 scope.go:117] "RemoveContainer" containerID="47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24"
Sep 10 16:39:59.271489 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:59.271471    2577 scope.go:117] "RemoveContainer" containerID="47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24"
Sep 10 16:39:59.271766 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:59.271749    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24\": container with ID starting with 47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24 not found: ID does not exist" containerID="47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24"
Sep 10 16:39:59.271837 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:39:59.271773    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24"} err="failed to get container status \"47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24\": rpc error: code = NotFound desc = could not find container \"47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24\": container with ID starting with 47479b9b88138f6d30a186a6c7451cf2fdda994468abf576b0d31af9acc7de24 not found: ID does not exist"
Sep 10 16:39:59.273589 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:39:59.273561    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7679-031ef0\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" pod="opendatahub/test-curl-7679-031ef0"
Sep 10 16:40:00.114007 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:00.113963    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b634c35-f1c5-4433-9345-d1e32f8f0f69" path="/var/lib/kubelet/pods/6b634c35-f1c5-4433-9345-d1e32f8f0f69/volumes"
Sep 10 16:40:00.463622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:00.463596    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:40:00.604593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:00.604558    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rdzz\" (UniqueName: \"kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz\") pod \"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6\" (UID: \"5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6\") "
Sep 10 16:40:00.606903 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:00.606868    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz" (OuterVolumeSpecName: "kube-api-access-8rdzz") pod "5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" (UID: "5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6"). InnerVolumeSpecName "kube-api-access-8rdzz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:40:00.705266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:00.705231    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rdzz\" (UniqueName: \"kubernetes.io/projected/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6-kube-api-access-8rdzz\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:40:01.277419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.277384    2577 generic.go:356] "Generic (PLEG): container finished" podID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" containerID="707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f" exitCode=137
Sep 10 16:40:01.277629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.277449    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:40:01.277629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.277488    2577 scope.go:117] "RemoveContainer" containerID="707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f"
Sep 10 16:40:01.279199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.279176    2577 generic.go:356] "Generic (PLEG): container finished" podID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerID="e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0" exitCode=0
Sep 10 16:40:01.279340 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.279256    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerDied","Data":"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"}
Sep 10 16:40:01.279705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.279689    2577 scope.go:117] "RemoveContainer" containerID="e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"
Sep 10 16:40:01.280824 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.280795    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:40:01.280909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.280868    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:40:01.280909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.280905    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:40:01.289557 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.289406    2577 scope.go:117] "RemoveContainer" containerID="707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f"
Sep 10 16:40:01.289747 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:40:01.289726    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f\": container with ID starting with 707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f not found: ID does not exist" containerID="707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f"
Sep 10 16:40:01.289812 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:01.289760    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f"} err="failed to get container status \"707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f\": rpc error: code = NotFound desc = could not find container \"707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f\": container with ID starting with 707a9061cc3e430557fd7349e30c45339784c0838d92b63cc72f6bc18a7b149f not found: ID does not exist"
Sep 10 16:40:01.304246 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:40:01.304211    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7679-09ce27\" is forbidden: User \"system:node:ip-10-0-136-40.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-136-40.ec2.internal' and this object" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" pod="opendatahub/test-curl-7679-09ce27"
Sep 10 16:40:02.114681 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:02.114644    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" path="/var/lib/kubelet/pods/5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6/volumes"
Sep 10 16:40:02.292135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:02.292065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerStarted","Data":"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"}
Sep 10 16:40:02.375626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:02.375538    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:40:02.376935 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:02.376903    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:02.377069 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:02.376953    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:03.376224 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:03.376184    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:03.376636 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:03.376255    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:04.376114 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:04.376082    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:04.376309 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:04.376140    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:05.375667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:05.375626    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:05.375868 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:05.375696    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:06.080134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:06.080099    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:40:06.375598 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:06.375567    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:06.375755 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:06.375622    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:07.375787 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:07.375741    2577 patch_prober.go:28] interesting pod/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 10 16:40:07.376330 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:07.375805    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 10 16:40:08.379420 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:08.379392    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:40:08.380491 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:08.380472    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:40:17.391724 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.391688    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:40:17.392388 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.392371    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" containerName="test-curl-7679-09ce27"
Sep 10 16:40:17.392388 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.392388    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" containerName="test-curl-7679-09ce27"
Sep 10 16:40:17.392515 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.392477    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e4ed03e-c8f8-4cfb-abd0-cfbebc7749e6" containerName="test-curl-7679-09ce27"
Sep 10 16:40:17.397221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.397201    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.400821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.400796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-cdfd80-serving-cert\""
Sep 10 16:40:17.413111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.413082    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:40:17.463022 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.462983    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.463207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.463037    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.463207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.463101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc8xd\" (UniqueName: \"kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.463207 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.463141    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.564474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.564440    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.564667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.564482    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.564667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.564535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc8xd\" (UniqueName: \"kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.564667 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.564571    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.565259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.565237    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.567039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.567018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.567106 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.567063    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.574772 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.574750    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc8xd\" (UniqueName: \"kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd\") pod \"maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:17.709926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:17.709837    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:18.005516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:18.005489    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:40:18.011241 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:40:18.011203    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8cd63c4_4c82_4de1_a975_416d2a592ad9.slice/crio-8f3681d869b3eb7e128d0c58d3285d64d4343e92dccf65e36206584730d3f6ee WatchSource:0}: Error finding container 8f3681d869b3eb7e128d0c58d3285d64d4343e92dccf65e36206584730d3f6ee: Status 404 returned error can't find the container with id 8f3681d869b3eb7e128d0c58d3285d64d4343e92dccf65e36206584730d3f6ee
Sep 10 16:40:18.412654 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:18.412611    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" event={"ID":"e8cd63c4-4c82-4de1-a975-416d2a592ad9","Type":"ContainerStarted","Data":"92307752e812eb652c302168d5e144b710a94058294a2544205311688aa9b452"}
Sep 10 16:40:18.413080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:18.412684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" event={"ID":"e8cd63c4-4c82-4de1-a975-416d2a592ad9","Type":"ContainerStarted","Data":"8f3681d869b3eb7e128d0c58d3285d64d4343e92dccf65e36206584730d3f6ee"}
Sep 10 16:40:18.413080 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:18.412757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:40:18.437742 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:18.437688    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" podStartSLOduration=1.154595557 podStartE2EDuration="1.437670446s" totalImagesPullingTime="283.074889ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:40:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:40:18.436679021 +0000 UTC m=+1390.838480147" watchObservedRunningTime="2026-09-10 16:40:18.437670446 +0000 UTC m=+1390.839471574"
Sep 10 16:40:24.424245 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:40:24.424209    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:41:00.247257 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.247170    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:41:00.690668 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.690359    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" podUID="cde1201f-e00d-4729-9c29-f9a370826d71" containerName="maas-api" containerID="cri-o://264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c" gracePeriod=30
Sep 10 16:41:00.962168 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.962139    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:41:00.996456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.996411    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs\") pod \"cde1201f-e00d-4729-9c29-f9a370826d71\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") "
Sep 10 16:41:00.996456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.996458    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6zg97\" (UniqueName: \"kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97\") pod \"cde1201f-e00d-4729-9c29-f9a370826d71\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") "
Sep 10 16:41:00.996707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.996536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca\") pod \"cde1201f-e00d-4729-9c29-f9a370826d71\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") "
Sep 10 16:41:00.996707 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.996573    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") pod \"cde1201f-e00d-4729-9c29-f9a370826d71\" (UID: \"cde1201f-e00d-4729-9c29-f9a370826d71\") "
Sep 10 16:41:00.997478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:00.997430    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cde1201f-e00d-4729-9c29-f9a370826d71" (UID: "cde1201f-e00d-4729-9c29-f9a370826d71"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:41:01.002682 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.001369    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cde1201f-e00d-4729-9c29-f9a370826d71" (UID: "cde1201f-e00d-4729-9c29-f9a370826d71"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:01.002997 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.002960    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97" (OuterVolumeSpecName: "kube-api-access-6zg97") pod "cde1201f-e00d-4729-9c29-f9a370826d71" (UID: "cde1201f-e00d-4729-9c29-f9a370826d71"). InnerVolumeSpecName "kube-api-access-6zg97". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:41:01.003855 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.003803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cde1201f-e00d-4729-9c29-f9a370826d71" (UID: "cde1201f-e00d-4729-9c29-f9a370826d71"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:01.097419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.097385    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:01.097419 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.097417    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6zg97\" (UniqueName: \"kubernetes.io/projected/cde1201f-e00d-4729-9c29-f9a370826d71-kube-api-access-6zg97\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:01.097626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.097432    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cde1201f-e00d-4729-9c29-f9a370826d71-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:01.097626 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.097444    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cde1201f-e00d-4729-9c29-f9a370826d71-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:01.695885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.695844    2577 generic.go:356] "Generic (PLEG): container finished" podID="cde1201f-e00d-4729-9c29-f9a370826d71" containerID="264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c" exitCode=0
Sep 10 16:41:01.696392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.695906    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"
Sep 10 16:41:01.696392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.695931    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" event={"ID":"cde1201f-e00d-4729-9c29-f9a370826d71","Type":"ContainerDied","Data":"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"}
Sep 10 16:41:01.696392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.695991    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94" event={"ID":"cde1201f-e00d-4729-9c29-f9a370826d71","Type":"ContainerDied","Data":"c26c34351b781112cd744c0f3a75580471d681f55a762b78f924131de189edae"}
Sep 10 16:41:01.696392 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.696020    2577 scope.go:117] "RemoveContainer" containerID="264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"
Sep 10 16:41:01.706633 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.706614    2577 scope.go:117] "RemoveContainer" containerID="264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"
Sep 10 16:41:01.706947 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:01.706927    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c\": container with ID starting with 264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c not found: ID does not exist" containerID="264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"
Sep 10 16:41:01.707026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.706956    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c"} err="failed to get container status \"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c\": rpc error: code = NotFound desc = could not find container \"264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c\": container with ID starting with 264a88c9169b80379d19b681188635550594e6b19c55f48ab0d54b217a5fc32c not found: ID does not exist"
Sep 10 16:41:01.738257 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.738225    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:41:01.753604 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:01.753570    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-fb3d69-86cc4679d4-xcq94"]
Sep 10 16:41:02.113344 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:02.113318    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde1201f-e00d-4729-9c29-f9a370826d71" path="/var/lib/kubelet/pods/cde1201f-e00d-4729-9c29-f9a370826d71/volumes"
Sep 10 16:41:10.509146 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:10.509112    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:41:10.509557 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:10.509351    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy" containerID="cri-o://eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12" gracePeriod=30
Sep 10 16:41:15.769027 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.769000    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:41:15.785454 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.785390    2577 generic.go:356] "Generic (PLEG): container finished" podID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerID="eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12" exitCode=0
Sep 10 16:41:15.785454 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.785435    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"
Sep 10 16:41:15.785610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.785477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerDied","Data":"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"}
Sep 10 16:41:15.785610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.785542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7" event={"ID":"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00","Type":"ContainerDied","Data":"e3d80fbf0961c12e44a1e3eedcd0b0f1734ea9ce27d909987c29cf0a03466adb"}
Sep 10 16:41:15.785610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.785577    2577 scope.go:117] "RemoveContainer" containerID="eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"
Sep 10 16:41:15.795991 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.795970    2577 scope.go:117] "RemoveContainer" containerID="e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"
Sep 10 16:41:15.805119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.805095    2577 scope.go:117] "RemoveContainer" containerID="eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"
Sep 10 16:41:15.805435 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:15.805414    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12\": container with ID starting with eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12 not found: ID does not exist" containerID="eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"
Sep 10 16:41:15.805525 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.805445    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12"} err="failed to get container status \"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12\": rpc error: code = NotFound desc = could not find container \"eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12\": container with ID starting with eb3a198ac49264e7b8415e05eaf04c89e46114a0f2a090321ee7b3686d554a12 not found: ID does not exist"
Sep 10 16:41:15.805525 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.805463    2577 scope.go:117] "RemoveContainer" containerID="e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"
Sep 10 16:41:15.805744 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:15.805727    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0\": container with ID starting with e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0 not found: ID does not exist" containerID="e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"
Sep 10 16:41:15.805792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.805756    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0"} err="failed to get container status \"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0\": rpc error: code = NotFound desc = could not find container \"e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0\": container with ID starting with e94e63d424c103fd77ad8460adc37a419dde28ea209ea9a6649c27e0f162fbb0 not found: ID does not exist"
Sep 10 16:41:15.837746 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837712    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.837959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837764    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.837959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.837959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.837959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837855    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.837959 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.837900    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.838260 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838026    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.838260 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838057    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.838260 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838132    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw2js\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js\") pod \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\" (UID: \"3db76c33-1c5c-4c8c-92a0-812c1aa2cb00\") "
Sep 10 16:41:15.838260 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838019    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:41:15.838482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838206    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:41:15.838482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838331    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:41:15.838482 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838447    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data" (OuterVolumeSpecName: "istio-data") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:41:15.838651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838556    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-data\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.838651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838574    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-credential-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.838651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838587    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istiod-ca-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.838651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838591    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:41:15.838651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.838601    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.840149 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.840112    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token" (OuterVolumeSpecName: "istio-token") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:41:15.840317 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.840269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js" (OuterVolumeSpecName: "kube-api-access-jw2js") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "kube-api-access-jw2js". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:41:15.840488 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.840469    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 16:41:15.840678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.840650    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" (UID: "3db76c33-1c5c-4c8c-92a0-812c1aa2cb00"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:41:15.940119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.940086    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-workload-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.940119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.940116    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw2js\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-kube-api-access-jw2js\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.940119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.940127    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.940381 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.940136    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-podinfo\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:15.940381 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:15.940144    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00-istio-envoy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:16.123455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:16.123422    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:41:16.131271 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:16.131238    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-fb3d69-openshift-default-69dc7759fb-v56j7"]
Sep 10 16:41:18.114474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:18.114443    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" path="/var/lib/kubelet/pods/3db76c33-1c5c-4c8c-92a0-812c1aa2cb00/volumes"
Sep 10 16:41:26.632159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:26.632125    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:41:26.632629 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:26.632370    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" podUID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" containerName="maas-api" containerID="cri-o://92307752e812eb652c302168d5e144b710a94058294a2544205311688aa9b452" gracePeriod=30
Sep 10 16:41:26.858950 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:26.858913    2577 generic.go:356] "Generic (PLEG): container finished" podID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" containerID="92307752e812eb652c302168d5e144b710a94058294a2544205311688aa9b452" exitCode=0
Sep 10 16:41:26.859111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:26.858983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" event={"ID":"e8cd63c4-4c82-4de1-a975-416d2a592ad9","Type":"ContainerDied","Data":"92307752e812eb652c302168d5e144b710a94058294a2544205311688aa9b452"}
Sep 10 16:41:26.889434 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:26.889366    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:41:27.058067 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.058035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls\") pod \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") "
Sep 10 16:41:27.058241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.058077    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca\") pod \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") "
Sep 10 16:41:27.058241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.058098    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs\") pod \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") "
Sep 10 16:41:27.058241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.058191    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pc8xd\" (UniqueName: \"kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd\") pod \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\" (UID: \"e8cd63c4-4c82-4de1-a975-416d2a592ad9\") "
Sep 10 16:41:27.058525 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.058494    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e8cd63c4-4c82-4de1-a975-416d2a592ad9" (UID: "e8cd63c4-4c82-4de1-a975-416d2a592ad9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:41:27.060385 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.060351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e8cd63c4-4c82-4de1-a975-416d2a592ad9" (UID: "e8cd63c4-4c82-4de1-a975-416d2a592ad9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:27.060385 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.060368    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd" (OuterVolumeSpecName: "kube-api-access-pc8xd") pod "e8cd63c4-4c82-4de1-a975-416d2a592ad9" (UID: "e8cd63c4-4c82-4de1-a975-416d2a592ad9"). InnerVolumeSpecName "kube-api-access-pc8xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:41:27.060385 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.060379    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e8cd63c4-4c82-4de1-a975-416d2a592ad9" (UID: "e8cd63c4-4c82-4de1-a975-416d2a592ad9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:27.159792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.159699    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:27.159792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.159732    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e8cd63c4-4c82-4de1-a975-416d2a592ad9-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:27.159792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.159744    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e8cd63c4-4c82-4de1-a975-416d2a592ad9-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:27.159792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.159753    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pc8xd\" (UniqueName: \"kubernetes.io/projected/e8cd63c4-4c82-4de1-a975-416d2a592ad9-kube-api-access-pc8xd\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:27.866892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.866861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn" event={"ID":"e8cd63c4-4c82-4de1-a975-416d2a592ad9","Type":"ContainerDied","Data":"8f3681d869b3eb7e128d0c58d3285d64d4343e92dccf65e36206584730d3f6ee"}
Sep 10 16:41:27.867315 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.866919    2577 scope.go:117] "RemoveContainer" containerID="92307752e812eb652c302168d5e144b710a94058294a2544205311688aa9b452"
Sep 10 16:41:27.867315 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.866876    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"
Sep 10 16:41:27.904365 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.904337    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:41:27.913838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:27.913809    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-cdfd80-5b7b5ff84-xmzsn"]
Sep 10 16:41:28.114314 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:28.114263    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" path="/var/lib/kubelet/pods/e8cd63c4-4c82-4de1-a975-416d2a592ad9/volumes"
Sep 10 16:41:39.060448 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060413    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060865    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060878    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060901    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060907    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060919    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde1201f-e00d-4729-9c29-f9a370826d71" containerName="maas-api"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060928    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde1201f-e00d-4729-9c29-f9a370826d71" containerName="maas-api"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060938    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" containerName="maas-api"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.060944    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" containerName="maas-api"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.061026    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8cd63c4-4c82-4de1-a975-416d2a592ad9" containerName="maas-api"
Sep 10 16:41:39.061029 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.061034    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.061043    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3db76c33-1c5c-4c8c-92a0-812c1aa2cb00" containerName="istio-proxy"
Sep 10 16:41:39.061593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.061050    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cde1201f-e00d-4729-9c29-f9a370826d71" containerName="maas-api"
Sep 10 16:41:39.064159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.064141    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.067356 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.067334    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-95d261-openshift-default-dockercfg-knb74\""
Sep 10 16:41:39.082734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.082706    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:41:39.173100 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173100 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173195    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.173352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.173344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c952\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274480    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274516 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274515    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c952\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274577    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274634    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274660    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.274765 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.275121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.274904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.275121 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.275033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.275234 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.275156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.275457 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.275433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.275747 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.275723    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.277189 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.277167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.277349 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.277329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.285370 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.285348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.285492 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.285472    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c952\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952\") pod \"e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") " pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.375129 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.375088    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:39.684435 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.684403    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:41:39.684583 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:41:39.684443    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceb0cb86_b640_4b46_b1ac_531e735b2b7f.slice/crio-a191699a882158482c66737ed08977903e901e318ee475dff012b618f7bbae8f WatchSource:0}: Error finding container a191699a882158482c66737ed08977903e901e318ee475dff012b618f7bbae8f: Status 404 returned error can't find the container with id a191699a882158482c66737ed08977903e901e318ee475dff012b618f7bbae8f
Sep 10 16:41:39.687030 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.686996    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:41:39.687132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.687061    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:41:39.687132 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.687090    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:41:39.947365 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.947261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerStarted","Data":"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"}
Sep 10 16:41:39.947365 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.947344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerStarted","Data":"a191699a882158482c66737ed08977903e901e318ee475dff012b618f7bbae8f"}
Sep 10 16:41:39.980547 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:39.980487    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podStartSLOduration=0.980468453 podStartE2EDuration="980.468453ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:41:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:41:39.976193824 +0000 UTC m=+1472.377994951" watchObservedRunningTime="2026-09-10 16:41:39.980468453 +0000 UTC m=+1472.382269581"
Sep 10 16:41:40.375955 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:40.375921    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:40.377559 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:40.377530    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:41:40.377691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:40.377594    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:41:41.379998 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:41.379967    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:41.962171 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:41.962137    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:41.963075 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:41.963053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:41:45.662937 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.662902    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:41:45.667124 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.667100    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.670910 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.670887    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-95d261-serving-cert\""
Sep 10 16:41:45.680402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.680376    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:41:45.739110 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.739078    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.739308 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.739127    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk9d7\" (UniqueName: \"kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.739308 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.739192    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.739308 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.739230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.840703 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.840666    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.840893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.840719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.840893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.840817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.840893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.840864    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk9d7\" (UniqueName: \"kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.841084 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:45.840959    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-serving-cert: secret "maas-api-e2e-ipp-a-95d261-serving-cert" not found
Sep 10 16:41:45.841084 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:45.841046    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls podName:b44ce3e5-3151-46da-8b9d-cae3fccd5215 nodeName:}" failed. No retries permitted until 2026-09-10 16:41:46.341025098 +0000 UTC m=+1478.742826225 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls") pod "maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" (UID: "b44ce3e5-3151-46da-8b9d-cae3fccd5215") : secret "maas-api-e2e-ipp-a-95d261-serving-cert" not found
Sep 10 16:41:45.841549 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.841525    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.843651 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.843630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:45.852039 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:45.852015    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk9d7\" (UniqueName: \"kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:46.345847 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.345749    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:46.348159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.348129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") pod \"maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:46.571220 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.571188    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:41:46.571458 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.571435    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" podUID="90435514-f49b-4e17-9e2f-a197c6f5b061" containerName="maas-api" containerID="cri-o://972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b" gracePeriod=30
Sep 10 16:41:46.578093 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.577971    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:46.830470 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.830445    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:41:46.877546 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.877209    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:41:46.882612 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:41:46.882581    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb44ce3e5_3151_46da_8b9d_cae3fccd5215.slice/crio-5c0acf860e1fe89517cafb19803b6f6d01e26df98c62af5d330dddddea4fc542 WatchSource:0}: Error finding container 5c0acf860e1fe89517cafb19803b6f6d01e26df98c62af5d330dddddea4fc542: Status 404 returned error can't find the container with id 5c0acf860e1fe89517cafb19803b6f6d01e26df98c62af5d330dddddea4fc542
Sep 10 16:41:46.951678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.951641    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls\") pod \"90435514-f49b-4e17-9e2f-a197c6f5b061\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") "
Sep 10 16:41:46.951867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.951739    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs\") pod \"90435514-f49b-4e17-9e2f-a197c6f5b061\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") "
Sep 10 16:41:46.951867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.951793    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpqmp\" (UniqueName: \"kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp\") pod \"90435514-f49b-4e17-9e2f-a197c6f5b061\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") "
Sep 10 16:41:46.951867 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.951835    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca\") pod \"90435514-f49b-4e17-9e2f-a197c6f5b061\" (UID: \"90435514-f49b-4e17-9e2f-a197c6f5b061\") "
Sep 10 16:41:46.952404 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.952371    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "90435514-f49b-4e17-9e2f-a197c6f5b061" (UID: "90435514-f49b-4e17-9e2f-a197c6f5b061"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:41:46.954074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.954045    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp" (OuterVolumeSpecName: "kube-api-access-gpqmp") pod "90435514-f49b-4e17-9e2f-a197c6f5b061" (UID: "90435514-f49b-4e17-9e2f-a197c6f5b061"). InnerVolumeSpecName "kube-api-access-gpqmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:41:46.954289 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.954249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "90435514-f49b-4e17-9e2f-a197c6f5b061" (UID: "90435514-f49b-4e17-9e2f-a197c6f5b061"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:46.954372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:46.954313    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "90435514-f49b-4e17-9e2f-a197c6f5b061" (UID: "90435514-f49b-4e17-9e2f-a197c6f5b061"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:41:47.004298 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.003800    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" event={"ID":"b44ce3e5-3151-46da-8b9d-cae3fccd5215","Type":"ContainerStarted","Data":"5c0acf860e1fe89517cafb19803b6f6d01e26df98c62af5d330dddddea4fc542"}
Sep 10 16:41:47.005971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.005937    2577 generic.go:356] "Generic (PLEG): container finished" podID="90435514-f49b-4e17-9e2f-a197c6f5b061" containerID="972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b" exitCode=0
Sep 10 16:41:47.006104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.005992    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"
Sep 10 16:41:47.006104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.006022    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" event={"ID":"90435514-f49b-4e17-9e2f-a197c6f5b061","Type":"ContainerDied","Data":"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"}
Sep 10 16:41:47.006104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.006090    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn" event={"ID":"90435514-f49b-4e17-9e2f-a197c6f5b061","Type":"ContainerDied","Data":"9403a43968d8c6913d7e82782dc306842ce26018e68bd12e2940c3928c1c4d34"}
Sep 10 16:41:47.006225 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.006120    2577 scope.go:117] "RemoveContainer" containerID="972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"
Sep 10 16:41:47.017036 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.017015    2577 scope.go:117] "RemoveContainer" containerID="972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"
Sep 10 16:41:47.017334 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:41:47.017314    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b\": container with ID starting with 972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b not found: ID does not exist" containerID="972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"
Sep 10 16:41:47.017391 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.017344    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b"} err="failed to get container status \"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b\": rpc error: code = NotFound desc = could not find container \"972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b\": container with ID starting with 972bd3609e7fa5dc176c7ba2e61f7191cbf63ffd81701ba5c628401d92d2cf1b not found: ID does not exist"
Sep 10 16:41:47.037783 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.037751    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:41:47.043030 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.043005    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-bd1ece-58ccd94c47-rr6mn"]
Sep 10 16:41:47.053330 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.053309    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpqmp\" (UniqueName: \"kubernetes.io/projected/90435514-f49b-4e17-9e2f-a197c6f5b061-kube-api-access-gpqmp\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:47.053330 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.053330    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90435514-f49b-4e17-9e2f-a197c6f5b061-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:47.053466 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.053341    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:47.053466 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:47.053351    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90435514-f49b-4e17-9e2f-a197c6f5b061-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:41:48.016404 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:48.016368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" event={"ID":"b44ce3e5-3151-46da-8b9d-cae3fccd5215","Type":"ContainerStarted","Data":"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"}
Sep 10 16:41:48.016855 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:48.016488    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:41:48.037564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:48.037516    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" podStartSLOduration=2.582064629 podStartE2EDuration="3.037498457s" totalImagesPullingTime="455.433828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:41:48.036086761 +0000 UTC m=+1480.437887887" watchObservedRunningTime="2026-09-10 16:41:48.037498457 +0000 UTC m=+1480.439299584"
Sep 10 16:41:48.113933 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:48.113900    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="90435514-f49b-4e17-9e2f-a197c6f5b061" path="/var/lib/kubelet/pods/90435514-f49b-4e17-9e2f-a197c6f5b061/volumes"
Sep 10 16:41:54.028433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:41:54.028402    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:42:02.107877 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.107841    2577 generic.go:356] "Generic (PLEG): container finished" podID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerID="78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d" exitCode=0
Sep 10 16:42:02.112522 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.112494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerDied","Data":"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"}
Sep 10 16:42:02.112880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.112865    2577 scope.go:117] "RemoveContainer" containerID="78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"
Sep 10 16:42:02.114085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.114055    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:42:02.114148 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.114132    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:42:02.114368 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:02.114168    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236216Ki","pods":"250"}
Sep 10 16:42:03.117547 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:03.117514    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerStarted","Data":"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"}
Sep 10 16:42:03.376150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:03.376062    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:42:03.376341 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:03.376306    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:03.376397 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:03.376373    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:04.375924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:04.375878    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:04.376326 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:04.375953    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:05.375673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:05.375638    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:05.375856 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:05.375696    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:05.569983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:05.569941    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:42:05.570423 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:05.570222    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" podUID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" containerName="maas-api" containerID="cri-o://cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47" gracePeriod=30
Sep 10 16:42:05.894904 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:05.889692    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:42:06.046601 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.046515    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") pod \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") "
Sep 10 16:42:06.046601 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.046578    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca\") pod \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") "
Sep 10 16:42:06.046794 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.046616    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs\") pod \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") "
Sep 10 16:42:06.046794 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.046667    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fq2l\" (UniqueName: \"kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l\") pod \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\" (UID: \"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1\") "
Sep 10 16:42:06.047118 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.046911    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" (UID: "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:42:06.047438 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.047401    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:42:06.048900 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.048875    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l" (OuterVolumeSpecName: "kube-api-access-7fq2l") pod "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" (UID: "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1"). InnerVolumeSpecName "kube-api-access-7fq2l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:42:06.049586 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.049564    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" (UID: "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:42:06.049713 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.049691    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" (UID: "0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:42:06.138750 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.138719    2577 generic.go:356] "Generic (PLEG): container finished" podID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" containerID="cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47" exitCode=0
Sep 10 16:42:06.138892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.138815    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"
Sep 10 16:42:06.138892 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.138816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" event={"ID":"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1","Type":"ContainerDied","Data":"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"}
Sep 10 16:42:06.138968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.138890    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk" event={"ID":"0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1","Type":"ContainerDied","Data":"11b7c825ed79daa158f12815e5b28b064409e5e13cb5eccde619a913e22bee71"}
Sep 10 16:42:06.138968 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.138921    2577 scope.go:117] "RemoveContainer" containerID="cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"
Sep 10 16:42:06.148695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.148671    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:42:06.148695 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.148695    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fq2l\" (UniqueName: \"kubernetes.io/projected/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-kube-api-access-7fq2l\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:42:06.148854 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.148704    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:42:06.149322 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.149301    2577 scope.go:117] "RemoveContainer" containerID="cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"
Sep 10 16:42:06.149610 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:42:06.149582    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47\": container with ID starting with cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47 not found: ID does not exist" containerID="cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"
Sep 10 16:42:06.149691 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.149618    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47"} err="failed to get container status \"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47\": rpc error: code = NotFound desc = could not find container \"cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47\": container with ID starting with cb849e2baec34c360d83102dac74582eb434686f786c7ac1c5dd02f12504bb47 not found: ID does not exist"
Sep 10 16:42:06.160362 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.160337    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:42:06.166429 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.166406    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-035bd5-78f9755cd7-qn9kk"]
Sep 10 16:42:06.375723 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.375663    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:06.375909 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:06.375749    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:07.376233 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:07.376196    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:07.376660 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:07.376269    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:08.117531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:08.117499    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" path="/var/lib/kubelet/pods/0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1/volumes"
Sep 10 16:42:08.218884 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:08.218851    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:42:08.225297 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:08.225256    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:42:08.375718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:08.375624    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body=
Sep 10 16:42:08.375718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:08.375694    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused"
Sep 10 16:42:09.380675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.380643    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:42:09.381098 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.380931    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:42:09.381761 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.381740    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:42:09.968705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.968674    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:42:09.969120 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969104    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" containerName="maas-api"
Sep 10 16:42:09.969212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969122    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" containerName="maas-api"
Sep 10 16:42:09.969212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969150    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="90435514-f49b-4e17-9e2f-a197c6f5b061" containerName="maas-api"
Sep 10 16:42:09.969212 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969156    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="90435514-f49b-4e17-9e2f-a197c6f5b061" containerName="maas-api"
Sep 10 16:42:09.969400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969229    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0fdf9fe6-0b34-4ea5-9c18-b81ed65640a1" containerName="maas-api"
Sep 10 16:42:09.969400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.969238    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="90435514-f49b-4e17-9e2f-a197c6f5b061" containerName="maas-api"
Sep 10 16:42:09.973908 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.973890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:09.978400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.978377    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-86d230-serving-cert\""
Sep 10 16:42:09.987433 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:09.987407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:42:10.086773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.086733    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.086926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.086874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.086926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.086898    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.086926 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.086923    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clhhm\" (UniqueName: \"kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.187776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.187738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.187776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.187780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.188058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.187803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clhhm\" (UniqueName: \"kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.188058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.187839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.188474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.188451    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.190340 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.190317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.190475 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.190364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.197614 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.197593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clhhm\" (UniqueName: \"kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm\") pod \"maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.285236 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.285143    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:10.576474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:10.576445    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:42:10.577388 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:42:10.577358    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39348503_c734_4dee_a86e_064c110eff65.slice/crio-5554b4445db2955ce033a8edc0a738f97bfc9b1a90362a12ceab59d022ebe240 WatchSource:0}: Error finding container 5554b4445db2955ce033a8edc0a738f97bfc9b1a90362a12ceab59d022ebe240: Status 404 returned error can't find the container with id 5554b4445db2955ce033a8edc0a738f97bfc9b1a90362a12ceab59d022ebe240
Sep 10 16:42:11.171606 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:11.171569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" event={"ID":"39348503-c734-4dee-a86e-064c110eff65","Type":"ContainerStarted","Data":"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"}
Sep 10 16:42:11.171799 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:11.171623    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" event={"ID":"39348503-c734-4dee-a86e-064c110eff65","Type":"ContainerStarted","Data":"5554b4445db2955ce033a8edc0a738f97bfc9b1a90362a12ceab59d022ebe240"}
Sep 10 16:42:11.171882 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:11.171818    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:42:11.194139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:11.194083    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" podStartSLOduration=1.748507213 podStartE2EDuration="2.194062651s" totalImagesPullingTime="445.555438ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:42:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:42:11.190469251 +0000 UTC m=+1503.592270378" watchObservedRunningTime="2026-09-10 16:42:11.194062651 +0000 UTC m=+1503.595863779"
Sep 10 16:42:17.183607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:42:17.183571    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:43:37.030941 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.030900    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:43:37.035017 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.034994    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.038617 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.038599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f9d55d-serving-cert\""
Sep 10 16:43:37.046461 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.046438    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:43:37.131338 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.131302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2d4d\" (UniqueName: \"kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.131533 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.131392    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.131533 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.131437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.131649 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.131540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.232983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.232948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2d4d\" (UniqueName: \"kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.233184 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.232993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.233184 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.233020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.233184 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.233079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.233184 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:43:37.233148    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-serving-cert: secret "maas-api-e2e-ipp-cleanup-f9d55d-serving-cert" not found
Sep 10 16:43:37.233445 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:43:37.233244    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls podName:9b5e4521-a152-4141-9a97-4d840a12cb21 nodeName:}" failed. No retries permitted until 2026-09-10 16:43:37.733220757 +0000 UTC m=+1590.135021869 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" (UID: "9b5e4521-a152-4141-9a97-4d840a12cb21") : secret "maas-api-e2e-ipp-cleanup-f9d55d-serving-cert" not found
Sep 10 16:43:37.233801 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.233779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.235480 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.235460    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.244219 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.244193    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2d4d\" (UniqueName: \"kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.738700 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.738655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.741739 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.741710    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:37.947337 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:37.947299    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:38.230253 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:38.230221    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:43:38.231712 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:43:38.231684    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5e4521_a152_4141_9a97_4d840a12cb21.slice/crio-b1e5781f73275cd08a28a981de199b0d2f622f9d9eff0c00738805f2a0e74621 WatchSource:0}: Error finding container b1e5781f73275cd08a28a981de199b0d2f622f9d9eff0c00738805f2a0e74621: Status 404 returned error can't find the container with id b1e5781f73275cd08a28a981de199b0d2f622f9d9eff0c00738805f2a0e74621
Sep 10 16:43:38.735151 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:38.735112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" event={"ID":"9b5e4521-a152-4141-9a97-4d840a12cb21","Type":"ContainerStarted","Data":"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"}
Sep 10 16:43:38.735378 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:38.735174    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" event={"ID":"9b5e4521-a152-4141-9a97-4d840a12cb21","Type":"ContainerStarted","Data":"b1e5781f73275cd08a28a981de199b0d2f622f9d9eff0c00738805f2a0e74621"}
Sep 10 16:43:38.735378 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:38.735311    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:43:38.754729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:38.754676    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" podStartSLOduration=1.352604585 podStartE2EDuration="1.75465673s" totalImagesPullingTime="402.052145ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:43:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:43:38.752907586 +0000 UTC m=+1591.154708727" watchObservedRunningTime="2026-09-10 16:43:38.75465673 +0000 UTC m=+1591.156457857"
Sep 10 16:43:44.747364 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:43:44.747335    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:44:15.482139 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.482044    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:44:15.482745 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.482329    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" podUID="9b5e4521-a152-4141-9a97-4d840a12cb21" containerName="maas-api" containerID="cri-o://eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248" gracePeriod=30
Sep 10 16:44:15.744456 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.744390    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:44:15.874607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.874576    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca\") pod \"9b5e4521-a152-4141-9a97-4d840a12cb21\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") "
Sep 10 16:44:15.874792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.874629    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") pod \"9b5e4521-a152-4141-9a97-4d840a12cb21\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") "
Sep 10 16:44:15.874792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.874680    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d2d4d\" (UniqueName: \"kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d\") pod \"9b5e4521-a152-4141-9a97-4d840a12cb21\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") "
Sep 10 16:44:15.874792 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.874738    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs\") pod \"9b5e4521-a152-4141-9a97-4d840a12cb21\" (UID: \"9b5e4521-a152-4141-9a97-4d840a12cb21\") "
Sep 10 16:44:15.875040 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.875011    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9b5e4521-a152-4141-9a97-4d840a12cb21" (UID: "9b5e4521-a152-4141-9a97-4d840a12cb21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:44:15.876880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.876842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d" (OuterVolumeSpecName: "kube-api-access-d2d4d") pod "9b5e4521-a152-4141-9a97-4d840a12cb21" (UID: "9b5e4521-a152-4141-9a97-4d840a12cb21"). InnerVolumeSpecName "kube-api-access-d2d4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:44:15.877083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.877062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9b5e4521-a152-4141-9a97-4d840a12cb21" (UID: "9b5e4521-a152-4141-9a97-4d840a12cb21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:44:15.877194 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.877159    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9b5e4521-a152-4141-9a97-4d840a12cb21" (UID: "9b5e4521-a152-4141-9a97-4d840a12cb21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:44:15.968252 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.968210    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b5e4521-a152-4141-9a97-4d840a12cb21" containerID="eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248" exitCode=0
Sep 10 16:44:15.968469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.968311    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"
Sep 10 16:44:15.968469 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.968454    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" event={"ID":"9b5e4521-a152-4141-9a97-4d840a12cb21","Type":"ContainerDied","Data":"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"}
Sep 10 16:44:15.968575 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.968545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g" event={"ID":"9b5e4521-a152-4141-9a97-4d840a12cb21","Type":"ContainerDied","Data":"b1e5781f73275cd08a28a981de199b0d2f622f9d9eff0c00738805f2a0e74621"}
Sep 10 16:44:15.968622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.968586    2577 scope.go:117] "RemoveContainer" containerID="eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"
Sep 10 16:44:15.976192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.976059    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d2d4d\" (UniqueName: \"kubernetes.io/projected/9b5e4521-a152-4141-9a97-4d840a12cb21-kube-api-access-d2d4d\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:15.976192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.976083    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:15.976192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.976098    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9b5e4521-a152-4141-9a97-4d840a12cb21-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:15.976192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.976111    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9b5e4521-a152-4141-9a97-4d840a12cb21-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:15.979050 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.979033    2577 scope.go:117] "RemoveContainer" containerID="eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"
Sep 10 16:44:15.979373 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:44:15.979352    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248\": container with ID starting with eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248 not found: ID does not exist" containerID="eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"
Sep 10 16:44:15.979436 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.979381    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248"} err="failed to get container status \"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248\": rpc error: code = NotFound desc = could not find container \"eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248\": container with ID starting with eac91204ab11cd1f185f198d30ad8a84995bb537b710e92fb94eb8bbbdef9248 not found: ID does not exist"
Sep 10 16:44:15.994610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.994529    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:44:15.997459 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:15.997433    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f9d55d-5cd646db79-pkf5g"]
Sep 10 16:44:16.113843 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:16.113809    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b5e4521-a152-4141-9a97-4d840a12cb21" path="/var/lib/kubelet/pods/9b5e4521-a152-4141-9a97-4d840a12cb21/volumes"
Sep 10 16:44:41.771317 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:41.771255    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:44:41.771894 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:41.771528    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" podUID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" containerName="maas-api" containerID="cri-o://769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2" gracePeriod=30
Sep 10 16:44:42.029218 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.029152    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:44:42.128210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.128174    2577 generic.go:356] "Generic (PLEG): container finished" podID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" containerID="769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2" exitCode=0
Sep 10 16:44:42.128390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.128228    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"
Sep 10 16:44:42.128390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.128264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" event={"ID":"b44ce3e5-3151-46da-8b9d-cae3fccd5215","Type":"ContainerDied","Data":"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"}
Sep 10 16:44:42.128390 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.128346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx" event={"ID":"b44ce3e5-3151-46da-8b9d-cae3fccd5215","Type":"ContainerDied","Data":"5c0acf860e1fe89517cafb19803b6f6d01e26df98c62af5d330dddddea4fc542"}
Sep 10 16:44:42.128520 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.128376    2577 scope.go:117] "RemoveContainer" containerID="769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"
Sep 10 16:44:42.133992 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.133970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk9d7\" (UniqueName: \"kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7\") pod \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") "
Sep 10 16:44:42.134127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.134009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") pod \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") "
Sep 10 16:44:42.134127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.134034    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca\") pod \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") "
Sep 10 16:44:42.134127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.134093    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs\") pod \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\" (UID: \"b44ce3e5-3151-46da-8b9d-cae3fccd5215\") "
Sep 10 16:44:42.134497 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.134469    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b44ce3e5-3151-46da-8b9d-cae3fccd5215" (UID: "b44ce3e5-3151-46da-8b9d-cae3fccd5215"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:44:42.136268 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.136242    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b44ce3e5-3151-46da-8b9d-cae3fccd5215" (UID: "b44ce3e5-3151-46da-8b9d-cae3fccd5215"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:44:42.136646 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.136619    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7" (OuterVolumeSpecName: "kube-api-access-sk9d7") pod "b44ce3e5-3151-46da-8b9d-cae3fccd5215" (UID: "b44ce3e5-3151-46da-8b9d-cae3fccd5215"). InnerVolumeSpecName "kube-api-access-sk9d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:44:42.136729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.136659    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b44ce3e5-3151-46da-8b9d-cae3fccd5215" (UID: "b44ce3e5-3151-46da-8b9d-cae3fccd5215"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:44:42.138148 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.138124    2577 scope.go:117] "RemoveContainer" containerID="769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"
Sep 10 16:44:42.138514 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:44:42.138486    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2\": container with ID starting with 769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2 not found: ID does not exist" containerID="769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"
Sep 10 16:44:42.138607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.138525    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2"} err="failed to get container status \"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2\": rpc error: code = NotFound desc = could not find container \"769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2\": container with ID starting with 769bc32327a5d1b85913e2977e35d9c24b8f8a942c974548c80454cf471b1ea2 not found: ID does not exist"
Sep 10 16:44:42.235845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.235808    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:42.235845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.235840    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk9d7\" (UniqueName: \"kubernetes.io/projected/b44ce3e5-3151-46da-8b9d-cae3fccd5215-kube-api-access-sk9d7\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:42.235845 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.235850    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b44ce3e5-3151-46da-8b9d-cae3fccd5215-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:42.236134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.235859    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b44ce3e5-3151-46da-8b9d-cae3fccd5215-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:44:42.452850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.452814    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:44:42.459156 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:42.459119    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-95d261-6c8769bc98-t6ztx"]
Sep 10 16:44:44.113873 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:44:44.113835    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" path="/var/lib/kubelet/pods/b44ce3e5-3151-46da-8b9d-cae3fccd5215/volumes"
Sep 10 16:45:31.976206 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:31.976127    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:45:31.976688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:31.976426    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy" containerID="cri-o://799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0" gracePeriod=30
Sep 10 16:45:37.237247 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.237190    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:45:37.356446 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356414    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356454    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356497    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c952\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356557    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356601    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356659 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356632    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356689    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356710    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data\") pod \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\" (UID: \"ceb0cb86-b640-4b46-b1ac-531e735b2b7f\") "
Sep 10 16:45:37.356972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356793    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:45:37.357134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356941    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:45:37.357134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356991    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:45:37.357134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357018    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:45:37.357134 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.356947    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:45:37.357401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357136    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-data\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.357401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357156    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istiod-ca-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.357401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357170    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-credential-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.357401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357182    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.357401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.357195    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-workload-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.358897 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.358871    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 16:45:37.358981 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.358952    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952" (OuterVolumeSpecName: "kube-api-access-2c952") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "kube-api-access-2c952". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:45:37.359186 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.359163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:45:37.359251 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.359232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ceb0cb86-b640-4b46-b1ac-531e735b2b7f" (UID: "ceb0cb86-b640-4b46-b1ac-531e735b2b7f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:45:37.458125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.458086    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-podinfo\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.458125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.458118    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.458125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.458126    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-istio-envoy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.458125 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.458134    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c952\" (UniqueName: \"kubernetes.io/projected/ceb0cb86-b640-4b46-b1ac-531e735b2b7f-kube-api-access-2c952\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:37.465204 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.465166    2577 generic.go:356] "Generic (PLEG): container finished" podID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerID="799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0" exitCode=0
Sep 10 16:45:37.465353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.465214    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"
Sep 10 16:45:37.465353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.465234    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerDied","Data":"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"}
Sep 10 16:45:37.465353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.465295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64" event={"ID":"ceb0cb86-b640-4b46-b1ac-531e735b2b7f","Type":"ContainerDied","Data":"a191699a882158482c66737ed08977903e901e318ee475dff012b618f7bbae8f"}
Sep 10 16:45:37.465353 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.465317    2577 scope.go:117] "RemoveContainer" containerID="799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"
Sep 10 16:45:37.482838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.482818    2577 scope.go:117] "RemoveContainer" containerID="78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"
Sep 10 16:45:37.494673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.494651    2577 scope.go:117] "RemoveContainer" containerID="799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"
Sep 10 16:45:37.495102 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:45:37.495075    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0\": container with ID starting with 799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0 not found: ID does not exist" containerID="799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"
Sep 10 16:45:37.495199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.495107    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0"} err="failed to get container status \"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0\": rpc error: code = NotFound desc = could not find container \"799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0\": container with ID starting with 799aec58ef9f43ef2feeff6d40405ab92f565b4bd400674cd456621ff04e6bd0 not found: ID does not exist"
Sep 10 16:45:37.495199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.495124    2577 scope.go:117] "RemoveContainer" containerID="78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"
Sep 10 16:45:37.495480 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:45:37.495457    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d\": container with ID starting with 78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d not found: ID does not exist" containerID="78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"
Sep 10 16:45:37.495554 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.495489    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d"} err="failed to get container status \"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d\": rpc error: code = NotFound desc = could not find container \"78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d\": container with ID starting with 78387e8b6529157b90a7d1213b3ba7e38db51f2905d6a6c7b9e21719645cf53d not found: ID does not exist"
Sep 10 16:45:37.497911 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.497891    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:45:37.501415 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:37.501395    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-95d261-openshift-default-67b45dd556-l5w64"]
Sep 10 16:45:38.114757 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:38.114725    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" path="/var/lib/kubelet/pods/ceb0cb86-b640-4b46-b1ac-531e735b2b7f/volumes"
Sep 10 16:45:42.783337 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:42.783297    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:45:42.783916 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:42.783587    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" podUID="39348503-c734-4dee-a86e-064c110eff65" containerName="maas-api" containerID="cri-o://1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7" gracePeriod=30
Sep 10 16:45:43.050105 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.050071    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:45:43.110041 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.109994    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls\") pod \"39348503-c734-4dee-a86e-064c110eff65\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") "
Sep 10 16:45:43.110211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.110080    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs\") pod \"39348503-c734-4dee-a86e-064c110eff65\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") "
Sep 10 16:45:43.110211 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.110176    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca\") pod \"39348503-c734-4dee-a86e-064c110eff65\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") "
Sep 10 16:45:43.110367 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.110208    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clhhm\" (UniqueName: \"kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm\") pod \"39348503-c734-4dee-a86e-064c110eff65\" (UID: \"39348503-c734-4dee-a86e-064c110eff65\") "
Sep 10 16:45:43.110688 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.110658    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39348503-c734-4dee-a86e-064c110eff65" (UID: "39348503-c734-4dee-a86e-064c110eff65"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:45:43.112729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.112686    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "39348503-c734-4dee-a86e-064c110eff65" (UID: "39348503-c734-4dee-a86e-064c110eff65"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:45:43.112851 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.112777    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39348503-c734-4dee-a86e-064c110eff65" (UID: "39348503-c734-4dee-a86e-064c110eff65"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:45:43.112917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.112855    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm" (OuterVolumeSpecName: "kube-api-access-clhhm") pod "39348503-c734-4dee-a86e-064c110eff65" (UID: "39348503-c734-4dee-a86e-064c110eff65"). InnerVolumeSpecName "kube-api-access-clhhm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:45:43.211698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.211664    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:43.211698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.211692    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39348503-c734-4dee-a86e-064c110eff65-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:43.211698 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.211702    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39348503-c734-4dee-a86e-064c110eff65-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:43.212007 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.211712    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clhhm\" (UniqueName: \"kubernetes.io/projected/39348503-c734-4dee-a86e-064c110eff65-kube-api-access-clhhm\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:45:43.503119 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.503081    2577 generic.go:356] "Generic (PLEG): container finished" podID="39348503-c734-4dee-a86e-064c110eff65" containerID="1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7" exitCode=0
Sep 10 16:45:43.503335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.503129    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"
Sep 10 16:45:43.503335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.503172    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" event={"ID":"39348503-c734-4dee-a86e-064c110eff65","Type":"ContainerDied","Data":"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"}
Sep 10 16:45:43.503335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.503226    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2" event={"ID":"39348503-c734-4dee-a86e-064c110eff65","Type":"ContainerDied","Data":"5554b4445db2955ce033a8edc0a738f97bfc9b1a90362a12ceab59d022ebe240"}
Sep 10 16:45:43.503335 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.503249    2577 scope.go:117] "RemoveContainer" containerID="1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"
Sep 10 16:45:43.513810 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.513789    2577 scope.go:117] "RemoveContainer" containerID="1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"
Sep 10 16:45:43.514099 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:45:43.514080    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7\": container with ID starting with 1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7 not found: ID does not exist" containerID="1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"
Sep 10 16:45:43.514163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.514109    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7"} err="failed to get container status \"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7\": rpc error: code = NotFound desc = could not find container \"1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7\": container with ID starting with 1e023855db07e09f06a2885eb3c57159173f73e87733d7461068a97159a240a7 not found: ID does not exist"
Sep 10 16:45:43.526969 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.526948    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:45:43.531397 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:43.531376    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-86d230-5bcc9dfcfc-m74n2"]
Sep 10 16:45:44.113203 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:45:44.113160    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39348503-c734-4dee-a86e-064c110eff65" path="/var/lib/kubelet/pods/39348503-c734-4dee-a86e-064c110eff65/volumes"
Sep 10 16:46:01.220558 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.220521    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:46:01.221005 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.220783    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" podUID="bfba5bce-2950-4220-840d-759aeb103c54" containerName="maas-api" containerID="cri-o://15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173" gracePeriod=30
Sep 10 16:46:01.476068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.476003    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:46:01.488453 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.488424    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca\") pod \"bfba5bce-2950-4220-840d-759aeb103c54\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") "
Sep 10 16:46:01.488624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.488484    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") pod \"bfba5bce-2950-4220-840d-759aeb103c54\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") "
Sep 10 16:46:01.488624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.488513    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rb4qx\" (UniqueName: \"kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx\") pod \"bfba5bce-2950-4220-840d-759aeb103c54\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") "
Sep 10 16:46:01.488624 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.488605    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs\") pod \"bfba5bce-2950-4220-840d-759aeb103c54\" (UID: \"bfba5bce-2950-4220-840d-759aeb103c54\") "
Sep 10 16:46:01.488856 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.488833    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bfba5bce-2950-4220-840d-759aeb103c54" (UID: "bfba5bce-2950-4220-840d-759aeb103c54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:46:01.489019 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.489000    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfba5bce-2950-4220-840d-759aeb103c54-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:01.490813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.490789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx" (OuterVolumeSpecName: "kube-api-access-rb4qx") pod "bfba5bce-2950-4220-840d-759aeb103c54" (UID: "bfba5bce-2950-4220-840d-759aeb103c54"). InnerVolumeSpecName "kube-api-access-rb4qx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:46:01.490813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.490794    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bfba5bce-2950-4220-840d-759aeb103c54" (UID: "bfba5bce-2950-4220-840d-759aeb103c54"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:01.491002 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.490981    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bfba5bce-2950-4220-840d-759aeb103c54" (UID: "bfba5bce-2950-4220-840d-759aeb103c54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:01.589878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.589838    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:01.589878 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.589878    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rb4qx\" (UniqueName: \"kubernetes.io/projected/bfba5bce-2950-4220-840d-759aeb103c54-kube-api-access-rb4qx\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:01.590127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.589894    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bfba5bce-2950-4220-840d-759aeb103c54-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:01.616605 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.616572    2577 generic.go:356] "Generic (PLEG): container finished" podID="bfba5bce-2950-4220-840d-759aeb103c54" containerID="15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173" exitCode=0
Sep 10 16:46:01.616804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.616618    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"
Sep 10 16:46:01.616804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.616636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" event={"ID":"bfba5bce-2950-4220-840d-759aeb103c54","Type":"ContainerDied","Data":"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"}
Sep 10 16:46:01.616804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.616690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m" event={"ID":"bfba5bce-2950-4220-840d-759aeb103c54","Type":"ContainerDied","Data":"ea114ceafd4325c1e057090e1db1e6dcb5f842b9cf0d56197b03073ed5a08b71"}
Sep 10 16:46:01.616804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.616718    2577 scope.go:117] "RemoveContainer" containerID="15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"
Sep 10 16:46:01.628101 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.628073    2577 scope.go:117] "RemoveContainer" containerID="15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"
Sep 10 16:46:01.628528 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:46:01.628504    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173\": container with ID starting with 15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173 not found: ID does not exist" containerID="15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"
Sep 10 16:46:01.628620 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.628537    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173"} err="failed to get container status \"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173\": rpc error: code = NotFound desc = could not find container \"15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173\": container with ID starting with 15f0592d191bd3a3d2b4faa35f5542c152f3301bd11a8d7ced944304db7ff173 not found: ID does not exist"
Sep 10 16:46:01.642840 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.642812    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:46:01.648921 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:01.648897    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1925d8-65b845fd8f-qrx9m"]
Sep 10 16:46:02.118380 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:02.118336    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfba5bce-2950-4220-840d-759aeb103c54" path="/var/lib/kubelet/pods/bfba5bce-2950-4220-840d-759aeb103c54/volumes"
Sep 10 16:46:08.817084 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:08.816996    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:46:08.817576 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:08.817240    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy" containerID="cri-o://67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778" gracePeriod=30
Sep 10 16:46:14.076588 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.076562    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:46:14.108738 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108662    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108738 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108754    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108832    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.108960 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108945    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.109227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.108984    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.109227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c4dc\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc\") pod \"eff30db2-8924-4ea2-91e5-93a0554eeece\" (UID: \"eff30db2-8924-4ea2-91e5-93a0554eeece\") "
Sep 10 16:46:14.109227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109081    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:46:14.109227 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109087    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:46:14.109483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109361    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eff30db2-8924-4ea2-91e5-93a0554eeece-istiod-ca-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.109483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109393    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-credential-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.109483 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109389    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data" (OuterVolumeSpecName: "istio-data") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:46:14.109642 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109595    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:46:14.109789 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.109766    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:46:14.111451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.111421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token" (OuterVolumeSpecName: "istio-token") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:46:14.111584 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.111490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc" (OuterVolumeSpecName: "kube-api-access-4c4dc") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "kube-api-access-4c4dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:46:14.111885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.111864    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:46:14.111885 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.111875    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eff30db2-8924-4ea2-91e5-93a0554eeece" (UID: "eff30db2-8924-4ea2-91e5-93a0554eeece"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 16:46:14.210529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210489    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-envoy\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210520    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-data\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210530    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-podinfo\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210529 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210540    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210552    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c4dc\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-kube-api-access-4c4dc\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210562    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eff30db2-8924-4ea2-91e5-93a0554eeece-istio-token\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.210800 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.210570    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eff30db2-8924-4ea2-91e5-93a0554eeece-workload-socket\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:14.697352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.697310    2577 generic.go:356] "Generic (PLEG): container finished" podID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerID="67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778" exitCode=0
Sep 10 16:46:14.697561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.697370    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"
Sep 10 16:46:14.697561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.697403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerDied","Data":"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"}
Sep 10 16:46:14.697561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.697497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c" event={"ID":"eff30db2-8924-4ea2-91e5-93a0554eeece","Type":"ContainerDied","Data":"a7ab4a97e7060f89c00d61b9d79e570988ca5476e8f4cec5639271216adc8a97"}
Sep 10 16:46:14.697561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.697527    2577 scope.go:117] "RemoveContainer" containerID="67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"
Sep 10 16:46:14.707059 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.707040    2577 scope.go:117] "RemoveContainer" containerID="55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"
Sep 10 16:46:14.715401 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.715383    2577 scope.go:117] "RemoveContainer" containerID="67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"
Sep 10 16:46:14.715669 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:46:14.715650    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778\": container with ID starting with 67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778 not found: ID does not exist" containerID="67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"
Sep 10 16:46:14.715718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.715678    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778"} err="failed to get container status \"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778\": rpc error: code = NotFound desc = could not find container \"67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778\": container with ID starting with 67d70eba67406fb76c908a2c4689fd3c7c98d9a081e295a64ee3d3b33a8c8778 not found: ID does not exist"
Sep 10 16:46:14.715718 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.715695    2577 scope.go:117] "RemoveContainer" containerID="55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"
Sep 10 16:46:14.715969 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:46:14.715951    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34\": container with ID starting with 55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34 not found: ID does not exist" containerID="55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"
Sep 10 16:46:14.716010 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.715985    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34"} err="failed to get container status \"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34\": rpc error: code = NotFound desc = could not find container \"55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34\": container with ID starting with 55dbe3083ec6d8551f6cac3a8d08b2d8b5efd66730d3800cc262fe4a80aeab34 not found: ID does not exist"
Sep 10 16:46:14.722961 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.722937    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:46:14.728591 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:14.728571    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-1925d8-openshift-default-5dd569bfd-hpw8c"]
Sep 10 16:46:16.114099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:16.114068    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" path="/var/lib/kubelet/pods/eff30db2-8924-4ea2-91e5-93a0554eeece/volumes"
Sep 10 16:46:19.711607 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:19.711574    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:46:19.712199 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:19.711887    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" podUID="de3b4eec-9782-442c-a68e-0fa9762cb778" containerName="maas-api" containerID="cri-o://dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27" gracePeriod=30
Sep 10 16:46:19.968253 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:19.968178    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:46:20.065027 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.064989    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs\") pod \"de3b4eec-9782-442c-a68e-0fa9762cb778\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") "
Sep 10 16:46:20.065238 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.065068    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca\") pod \"de3b4eec-9782-442c-a68e-0fa9762cb778\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") "
Sep 10 16:46:20.065238 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.065154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mswm8\" (UniqueName: \"kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8\") pod \"de3b4eec-9782-442c-a68e-0fa9762cb778\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") "
Sep 10 16:46:20.065238 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.065211    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls\") pod \"de3b4eec-9782-442c-a68e-0fa9762cb778\" (UID: \"de3b4eec-9782-442c-a68e-0fa9762cb778\") "
Sep 10 16:46:20.065557 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.065532    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "de3b4eec-9782-442c-a68e-0fa9762cb778" (UID: "de3b4eec-9782-442c-a68e-0fa9762cb778"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:46:20.065656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.065641    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/de3b4eec-9782-442c-a68e-0fa9762cb778-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:20.067473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.067441    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8" (OuterVolumeSpecName: "kube-api-access-mswm8") pod "de3b4eec-9782-442c-a68e-0fa9762cb778" (UID: "de3b4eec-9782-442c-a68e-0fa9762cb778"). InnerVolumeSpecName "kube-api-access-mswm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:46:20.067573 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.067534    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "de3b4eec-9782-442c-a68e-0fa9762cb778" (UID: "de3b4eec-9782-442c-a68e-0fa9762cb778"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:20.067621 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.067598    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "de3b4eec-9782-442c-a68e-0fa9762cb778" (UID: "de3b4eec-9782-442c-a68e-0fa9762cb778"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:20.166838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.166804    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mswm8\" (UniqueName: \"kubernetes.io/projected/de3b4eec-9782-442c-a68e-0fa9762cb778-kube-api-access-mswm8\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:20.166838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.166839    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:20.167041 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.166850    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/de3b4eec-9782-442c-a68e-0fa9762cb778-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:20.741578 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.741541    2577 generic.go:356] "Generic (PLEG): container finished" podID="de3b4eec-9782-442c-a68e-0fa9762cb778" containerID="dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27" exitCode=0
Sep 10 16:46:20.742074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.741601    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"
Sep 10 16:46:20.742074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.741630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" event={"ID":"de3b4eec-9782-442c-a68e-0fa9762cb778","Type":"ContainerDied","Data":"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"}
Sep 10 16:46:20.742074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.741730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv" event={"ID":"de3b4eec-9782-442c-a68e-0fa9762cb778","Type":"ContainerDied","Data":"30c5d6695fa26e5e3a24505d62bdf3e1984fdad26e15e8074f779161100577a9"}
Sep 10 16:46:20.742074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.741762    2577 scope.go:117] "RemoveContainer" containerID="dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"
Sep 10 16:46:20.751496 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.751472    2577 scope.go:117] "RemoveContainer" containerID="dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"
Sep 10 16:46:20.751921 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:46:20.751884    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27\": container with ID starting with dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27 not found: ID does not exist" containerID="dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"
Sep 10 16:46:20.751994 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.751942    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27"} err="failed to get container status \"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27\": rpc error: code = NotFound desc = could not find container \"dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27\": container with ID starting with dd5de52013f6a7a8fff2dd74024460164e4e2584ac99fc0f9a1d1f9f3f6f0f27 not found: ID does not exist"
Sep 10 16:46:20.765014 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.764989    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:46:20.770062 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:20.770039    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-3ccbd4-69957d7dd6-2wqcv"]
Sep 10 16:46:22.113527 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:22.113485    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de3b4eec-9782-442c-a68e-0fa9762cb778" path="/var/lib/kubelet/pods/de3b4eec-9782-442c-a68e-0fa9762cb778/volumes"
Sep 10 16:46:30.161236 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.161197    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:46:30.161682 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.161414    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" podUID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" containerName="manager" containerID="cri-o://764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52" gracePeriod=30
Sep 10 16:46:30.422448 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.422382    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:46:30.563229 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.563190    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs\") pod \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") "
Sep 10 16:46:30.563229 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.563236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sg67d\" (UniqueName: \"kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d\") pod \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") "
Sep 10 16:46:30.563476 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.563260    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert\") pod \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\" (UID: \"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844\") "
Sep 10 16:46:30.565394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.565362    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d" (OuterVolumeSpecName: "kube-api-access-sg67d") pod "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" (UID: "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844"). InnerVolumeSpecName "kube-api-access-sg67d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:46:30.565394 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.565378    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" (UID: "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:30.565543 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.565450    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" (UID: "7a3f10bd-8e11-4ffc-8392-d8bb3cc84844"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:46:30.664228 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.664185    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:30.664228 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.664223    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sg67d\" (UniqueName: \"kubernetes.io/projected/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-kube-api-access-sg67d\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:30.664228 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.664234    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844-webhook-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:46:30.803842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.803737    2577 generic.go:356] "Generic (PLEG): container finished" podID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" containerID="764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52" exitCode=0
Sep 10 16:46:30.803842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.803786    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-hzntg"
Sep 10 16:46:30.803842 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.803825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" event={"ID":"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844","Type":"ContainerDied","Data":"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"}
Sep 10 16:46:30.804142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.803887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-hzntg" event={"ID":"7a3f10bd-8e11-4ffc-8392-d8bb3cc84844","Type":"ContainerDied","Data":"4f39de27f611d45f29674d0cdc442fdd7519976ae2e89b010ff08207a505db9c"}
Sep 10 16:46:30.804142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.803924    2577 scope.go:117] "RemoveContainer" containerID="764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"
Sep 10 16:46:30.814563 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.814542    2577 scope.go:117] "RemoveContainer" containerID="764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"
Sep 10 16:46:30.814836 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:46:30.814815    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52\": container with ID starting with 764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52 not found: ID does not exist" containerID="764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"
Sep 10 16:46:30.814917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.814850    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52"} err="failed to get container status \"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52\": rpc error: code = NotFound desc = could not find container \"764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52\": container with ID starting with 764e8fe98952e7ec23f7a5337e0a9a874697c29154072ac41e7e6f3c854e3b52 not found: ID does not exist"
Sep 10 16:46:30.831552 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.831517    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:46:30.833944 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:30.833921    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-hzntg"]
Sep 10 16:46:32.113400 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:46:32.113359    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" path="/var/lib/kubelet/pods/7a3f10bd-8e11-4ffc-8392-d8bb3cc84844/volumes"
Sep 10 16:47:08.257812 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:08.257782    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:47:08.267763 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:08.267739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:47:31.642214 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642176    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:31.642613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642588    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:31.642613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642601    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:31.642613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642607    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:31.642613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642612    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642622    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfba5bce-2950-4220-840d-759aeb103c54" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642627    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfba5bce-2950-4220-840d-759aeb103c54" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642634    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39348503-c734-4dee-a86e-064c110eff65" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642639    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="39348503-c734-4dee-a86e-064c110eff65" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642648    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642653    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642658    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642663    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642668    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b5e4521-a152-4141-9a97-4d840a12cb21" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642673    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b5e4521-a152-4141-9a97-4d840a12cb21" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642679    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" containerName="manager"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642684    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" containerName="manager"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642690    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de3b4eec-9782-442c-a68e-0fa9762cb778" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642695    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="de3b4eec-9782-442c-a68e-0fa9762cb778" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642702    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" containerName="maas-api"
Sep 10 16:47:31.642748 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642707    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" containerName="maas-api"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642766    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642774    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b5e4521-a152-4141-9a97-4d840a12cb21" containerName="maas-api"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="de3b4eec-9782-442c-a68e-0fa9762cb778" containerName="maas-api"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642786    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b44ce3e5-3151-46da-8b9d-cae3fccd5215" containerName="maas-api"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642792    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7a3f10bd-8e11-4ffc-8392-d8bb3cc84844" containerName="manager"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642797    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642803    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="39348503-c734-4dee-a86e-064c110eff65" containerName="maas-api"
Sep 10 16:47:31.643246 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.642810    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfba5bce-2950-4220-840d-759aeb103c54" containerName="maas-api"
Sep 10 16:47:31.645880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.645859    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.648813 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.648787    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:31.649930 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.649901    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:47:31.649930 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.649919    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:47:31.650112 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.649972    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tl6vz\""
Sep 10 16:47:31.650112 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.649903    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:47:31.804554 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.804524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.804729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.804641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqz5g\" (UniqueName: \"kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.804729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.804698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.905910 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.905820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.905910 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.905901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqz5g\" (UniqueName: \"kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.906135 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.905927    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.908355 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.908325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.908478 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.908455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.915202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.915181    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqz5g\" (UniqueName: \"kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g\") pod \"maas-controller-7bb45c94d6-wctvt\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") " pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:31.959405 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:31.959370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:32.234133 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:32.234102    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:32.235765 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:47:32.235735    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod448090a9_ff42_4ef7_802b_e7c0758a0683.slice/crio-f5f6cd7a523524d55e1fe6cb8f3d8154802cef4f21d8df8b8cd1b691deda9384 WatchSource:0}: Error finding container f5f6cd7a523524d55e1fe6cb8f3d8154802cef4f21d8df8b8cd1b691deda9384: Status 404 returned error can't find the container with id f5f6cd7a523524d55e1fe6cb8f3d8154802cef4f21d8df8b8cd1b691deda9384
Sep 10 16:47:32.237061 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:32.237042    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:47:33.153534 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:33.153497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" event={"ID":"448090a9-ff42-4ef7-802b-e7c0758a0683","Type":"ContainerStarted","Data":"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"}
Sep 10 16:47:33.153958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:33.153554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" event={"ID":"448090a9-ff42-4ef7-802b-e7c0758a0683","Type":"ContainerStarted","Data":"f5f6cd7a523524d55e1fe6cb8f3d8154802cef4f21d8df8b8cd1b691deda9384"}
Sep 10 16:47:33.153958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:33.153582    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:33.177522 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:33.177467    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" podStartSLOduration=1.642334781 podStartE2EDuration="2.177451039s" totalImagesPullingTime="535.116258ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:47:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:47:33.175202006 +0000 UTC m=+1825.577003135" watchObservedRunningTime="2026-09-10 16:47:33.177451039 +0000 UTC m=+1825.579252166"
Sep 10 16:47:44.162085 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:44.162052    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:53.480210 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.480171    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:53.480686 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.480451    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" podUID="448090a9-ff42-4ef7-802b-e7c0758a0683" containerName="manager" containerID="cri-o://6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d" gracePeriod=30
Sep 10 16:47:53.736972 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.736914    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:53.785168 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.785135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs\") pod \"448090a9-ff42-4ef7-802b-e7c0758a0683\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") "
Sep 10 16:47:53.785376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.785228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqz5g\" (UniqueName: \"kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g\") pod \"448090a9-ff42-4ef7-802b-e7c0758a0683\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") "
Sep 10 16:47:53.785376 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.785289    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert\") pod \"448090a9-ff42-4ef7-802b-e7c0758a0683\" (UID: \"448090a9-ff42-4ef7-802b-e7c0758a0683\") "
Sep 10 16:47:53.787924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.787872    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "448090a9-ff42-4ef7-802b-e7c0758a0683" (UID: "448090a9-ff42-4ef7-802b-e7c0758a0683"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:47:53.787924 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.787905    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g" (OuterVolumeSpecName: "kube-api-access-vqz5g") pod "448090a9-ff42-4ef7-802b-e7c0758a0683" (UID: "448090a9-ff42-4ef7-802b-e7c0758a0683"). InnerVolumeSpecName "kube-api-access-vqz5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:47:53.788081 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.787917    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "448090a9-ff42-4ef7-802b-e7c0758a0683" (UID: "448090a9-ff42-4ef7-802b-e7c0758a0683"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:47:53.886593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.886554    2577 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-webhook-cert\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:47:53.886593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.886587    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/448090a9-ff42-4ef7-802b-e7c0758a0683-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:47:53.886593 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:53.886597    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqz5g\" (UniqueName: \"kubernetes.io/projected/448090a9-ff42-4ef7-802b-e7c0758a0683-kube-api-access-vqz5g\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:47:54.274076 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.274040    2577 generic.go:356] "Generic (PLEG): container finished" podID="448090a9-ff42-4ef7-802b-e7c0758a0683" containerID="6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d" exitCode=0
Sep 10 16:47:54.274254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.274086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-wctvt"
Sep 10 16:47:54.274254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.274121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" event={"ID":"448090a9-ff42-4ef7-802b-e7c0758a0683","Type":"ContainerDied","Data":"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"}
Sep 10 16:47:54.274254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.274169    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-wctvt" event={"ID":"448090a9-ff42-4ef7-802b-e7c0758a0683","Type":"ContainerDied","Data":"f5f6cd7a523524d55e1fe6cb8f3d8154802cef4f21d8df8b8cd1b691deda9384"}
Sep 10 16:47:54.274254 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.274193    2577 scope.go:117] "RemoveContainer" containerID="6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"
Sep 10 16:47:54.283835 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.283802    2577 scope.go:117] "RemoveContainer" containerID="6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"
Sep 10 16:47:54.284145 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:47:54.284123    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d\": container with ID starting with 6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d not found: ID does not exist" containerID="6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"
Sep 10 16:47:54.284232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.284154    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d"} err="failed to get container status \"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d\": rpc error: code = NotFound desc = could not find container \"6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d\": container with ID starting with 6f1a16b5b1de96f0bdd95e7ed4ac87f311e29d8abb0887b9522f1203fa77397d not found: ID does not exist"
Sep 10 16:47:54.296086 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.296059    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:54.299987 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:54.299957    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-wctvt"]
Sep 10 16:47:55.076961 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.076928    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-xtvmv"]
Sep 10 16:47:55.077352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.077333    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="448090a9-ff42-4ef7-802b-e7c0758a0683" containerName="manager"
Sep 10 16:47:55.077352 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.077345    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="448090a9-ff42-4ef7-802b-e7c0758a0683" containerName="manager"
Sep 10 16:47:55.077439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.077415    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ceb0cb86-b640-4b46-b1ac-531e735b2b7f" containerName="istio-proxy"
Sep 10 16:47:55.077439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.077425    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="448090a9-ff42-4ef7-802b-e7c0758a0683" containerName="manager"
Sep 10 16:47:55.077439 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.077433    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="eff30db2-8924-4ea2-91e5-93a0554eeece" containerName="istio-proxy"
Sep 10 16:47:55.081669 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.081650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.085661 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.085640    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 16:47:55.085797 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.085714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 16:47:55.085797 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.085757    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tl6vz\""
Sep 10 16:47:55.085941 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.085757    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 16:47:55.094008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.093984    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-xtvmv"]
Sep 10 16:47:55.097232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.097204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-metrics-certs\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.097402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.097294    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-webhook-cert\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.097402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.097382    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvw44\" (UniqueName: \"kubernetes.io/projected/467233d7-5c3e-48c1-9a6c-d5243ec980ea-kube-api-access-lvw44\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.198097 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.198052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-metrics-certs\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.198345 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.198117    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-webhook-cert\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.198345 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.198158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvw44\" (UniqueName: \"kubernetes.io/projected/467233d7-5c3e-48c1-9a6c-d5243ec980ea-kube-api-access-lvw44\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.200710 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.200680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-webhook-cert\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.200816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.200718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/467233d7-5c3e-48c1-9a6c-d5243ec980ea-metrics-certs\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.207564 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.207543    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvw44\" (UniqueName: \"kubernetes.io/projected/467233d7-5c3e-48c1-9a6c-d5243ec980ea-kube-api-access-lvw44\") pod \"maas-controller-7bb45c94d6-xtvmv\" (UID: \"467233d7-5c3e-48c1-9a6c-d5243ec980ea\") " pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.395213 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.395163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:55.669853 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:55.669821    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7bb45c94d6-xtvmv"]
Sep 10 16:47:55.670895 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:47:55.670864    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467233d7_5c3e_48c1_9a6c_d5243ec980ea.slice/crio-dd3a3f8e58ce5cd6f1fb3807bd3ceb1e304e3190d9c66f019980c7242c3416a5 WatchSource:0}: Error finding container dd3a3f8e58ce5cd6f1fb3807bd3ceb1e304e3190d9c66f019980c7242c3416a5: Status 404 returned error can't find the container with id dd3a3f8e58ce5cd6f1fb3807bd3ceb1e304e3190d9c66f019980c7242c3416a5
Sep 10 16:47:56.112820 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:56.112786    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="448090a9-ff42-4ef7-802b-e7c0758a0683" path="/var/lib/kubelet/pods/448090a9-ff42-4ef7-802b-e7c0758a0683/volumes"
Sep 10 16:47:56.295416 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:56.295301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv" event={"ID":"467233d7-5c3e-48c1-9a6c-d5243ec980ea","Type":"ContainerStarted","Data":"0752fd5c7f3ebd39e95d8590ef857a986f21192e45b950b8cac0c232992de638"}
Sep 10 16:47:56.295416 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:56.295401    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv" event={"ID":"467233d7-5c3e-48c1-9a6c-d5243ec980ea","Type":"ContainerStarted","Data":"dd3a3f8e58ce5cd6f1fb3807bd3ceb1e304e3190d9c66f019980c7242c3416a5"}
Sep 10 16:47:56.295680 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:56.295433    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:47:56.317802 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:47:56.317743    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv" podStartSLOduration=0.950032802 podStartE2EDuration="1.317723296s" totalImagesPullingTime="367.690494ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:47:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:47:56.313564586 +0000 UTC m=+1848.715365713" watchObservedRunningTime="2026-09-10 16:47:56.317723296 +0000 UTC m=+1848.719524425"
Sep 10 16:48:07.304473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:48:07.304438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7bb45c94d6-xtvmv"
Sep 10 16:49:10.413002 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.412962    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:49:10.413504 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.413158    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" podUID="88d5dc12-8352-415a-bc32-2590a7da7c7c" containerName="manager" containerID="cri-o://8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c" gracePeriod=10
Sep 10 16:49:10.667358 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.667300    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:49:10.728528 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.728491    2577 generic.go:356] "Generic (PLEG): container finished" podID="88d5dc12-8352-415a-bc32-2590a7da7c7c" containerID="8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c" exitCode=0
Sep 10 16:49:10.728711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.728538    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"
Sep 10 16:49:10.728711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.728575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" event={"ID":"88d5dc12-8352-415a-bc32-2590a7da7c7c","Type":"ContainerDied","Data":"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"}
Sep 10 16:49:10.728711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.728636    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5" event={"ID":"88d5dc12-8352-415a-bc32-2590a7da7c7c","Type":"ContainerDied","Data":"5db6889cf366d654f30337a2b6e2c22ca21f81895d4bb75109b1943526b08501"}
Sep 10 16:49:10.728711 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.728670    2577 scope.go:117] "RemoveContainer" containerID="8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"
Sep 10 16:49:10.740473 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.740455    2577 scope.go:117] "RemoveContainer" containerID="8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"
Sep 10 16:49:10.740749 ip-10-0-136-40 kubenswrapper[2577]: E0910 16:49:10.740728    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c\": container with ID starting with 8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c not found: ID does not exist" containerID="8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"
Sep 10 16:49:10.740832 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.740758    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c"} err="failed to get container status \"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c\": rpc error: code = NotFound desc = could not find container \"8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c\": container with ID starting with 8578ae2d03d06c210e5761aeaf4f7681b1861adbb78a43297827a0b1fd70a01c not found: ID does not exist"
Sep 10 16:49:10.774833 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.774805    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume\") pod \"88d5dc12-8352-415a-bc32-2590a7da7c7c\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") "
Sep 10 16:49:10.775006 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.774895    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crslp\" (UniqueName: \"kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp\") pod \"88d5dc12-8352-415a-bc32-2590a7da7c7c\" (UID: \"88d5dc12-8352-415a-bc32-2590a7da7c7c\") "
Sep 10 16:49:10.775232 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.775210    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "88d5dc12-8352-415a-bc32-2590a7da7c7c" (UID: "88d5dc12-8352-415a-bc32-2590a7da7c7c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 16:49:10.776902 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.776877    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp" (OuterVolumeSpecName: "kube-api-access-crslp") pod "88d5dc12-8352-415a-bc32-2590a7da7c7c" (UID: "88d5dc12-8352-415a-bc32-2590a7da7c7c"). InnerVolumeSpecName "kube-api-access-crslp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:49:10.876402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.876365    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88d5dc12-8352-415a-bc32-2590a7da7c7c-extensions-socket-volume\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:49:10.876402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:10.876394    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crslp\" (UniqueName: \"kubernetes.io/projected/88d5dc12-8352-415a-bc32-2590a7da7c7c-kube-api-access-crslp\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:49:11.054033 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:11.053999    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:49:11.058055 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:11.058035    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-fhlr5"]
Sep 10 16:49:12.113662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:49:12.113627    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88d5dc12-8352-415a-bc32-2590a7da7c7c" path="/var/lib/kubelet/pods/88d5dc12-8352-415a-bc32-2590a7da7c7c/volumes"
Sep 10 16:50:11.103159 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.103125    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"]
Sep 10 16:50:11.103585 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.103535    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88d5dc12-8352-415a-bc32-2590a7da7c7c" containerName="manager"
Sep 10 16:50:11.103585 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.103547    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88d5dc12-8352-415a-bc32-2590a7da7c7c" containerName="manager"
Sep 10 16:50:11.103662 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.103609    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88d5dc12-8352-415a-bc32-2590a7da7c7c" containerName="manager"
Sep 10 16:50:11.107893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.107869    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.111182 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.111156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sq4hz\""
Sep 10 16:50:11.122027 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.122000    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"]
Sep 10 16:50:11.189716 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.189675    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pbns\" (UniqueName: \"kubernetes.io/projected/27888c2d-aa62-43ab-b234-7398d2ace5ec-kube-api-access-7pbns\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.189943 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.189730    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27888c2d-aa62-43ab-b234-7398d2ace5ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.291192 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.291147    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pbns\" (UniqueName: \"kubernetes.io/projected/27888c2d-aa62-43ab-b234-7398d2ace5ec-kube-api-access-7pbns\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.291399 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.291208    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27888c2d-aa62-43ab-b234-7398d2ace5ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.291656 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.291634    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/27888c2d-aa62-43ab-b234-7398d2ace5ec-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.303506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.303481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pbns\" (UniqueName: \"kubernetes.io/projected/27888c2d-aa62-43ab-b234-7398d2ace5ec-kube-api-access-7pbns\") pod \"kuadrant-operator-controller-manager-5769f99d9f-crzrw\" (UID: \"27888c2d-aa62-43ab-b234-7398d2ace5ec\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.418163 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.418134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:11.708935 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:11.708908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"]
Sep 10 16:50:11.711680 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:50:11.711651    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27888c2d_aa62_43ab_b234_7398d2ace5ec.slice/crio-81a133ef452595749f6b3239f9430da8e7e20a0a71407bcdde228071d3da8067 WatchSource:0}: Error finding container 81a133ef452595749f6b3239f9430da8e7e20a0a71407bcdde228071d3da8067: Status 404 returned error can't find the container with id 81a133ef452595749f6b3239f9430da8e7e20a0a71407bcdde228071d3da8067
Sep 10 16:50:12.080506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:12.080412    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw" event={"ID":"27888c2d-aa62-43ab-b234-7398d2ace5ec","Type":"ContainerStarted","Data":"e23994f9803473e311de880fd65caa61df11e1a70b604d8a152450e2f3b09a76"}
Sep 10 16:50:12.080506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:12.080470    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:50:12.080506 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:12.080490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw" event={"ID":"27888c2d-aa62-43ab-b234-7398d2ace5ec","Type":"ContainerStarted","Data":"81a133ef452595749f6b3239f9430da8e7e20a0a71407bcdde228071d3da8067"}
Sep 10 16:50:12.110518 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:12.110469    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw" podStartSLOduration=1.110451826 podStartE2EDuration="1.110451826s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:50:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:50:12.109473832 +0000 UTC m=+1984.511274959" watchObservedRunningTime="2026-09-10 16:50:12.110451826 +0000 UTC m=+1984.512252952"
Sep 10 16:50:23.088424 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:50:23.088390    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-crzrw"
Sep 10 16:52:06.558733 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.558698    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"]
Sep 10 16:52:06.562226 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.562207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.574432 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.574406    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"]
Sep 10 16:52:06.641283 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.641250    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4642z\" (UniqueName: \"kubernetes.io/projected/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-kube-api-access-4642z\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.641451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.641371    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-maas-api-tls\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.641451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.641409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-openshift-service-ca\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.641451 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.641437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-metrics-certs\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.741971 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.741933    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4642z\" (UniqueName: \"kubernetes.io/projected/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-kube-api-access-4642z\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.742150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.741992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-maas-api-tls\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.742150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.742020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-openshift-service-ca\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.742150 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.742038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-metrics-certs\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.742740 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.742714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-openshift-service-ca\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.744603 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.744579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-metrics-certs\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.744917 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.744897    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-maas-api-tls\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.750561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.750538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4642z\" (UniqueName: \"kubernetes.io/projected/4f9e6567-4107-49d3-b9ab-1fc9449c9a61-kube-api-access-4642z\") pod \"maas-api-ddc7f4579-pwv55\" (UID: \"4f9e6567-4107-49d3-b9ab-1fc9449c9a61\") " pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:06.873146 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:06.873106    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:07.148957 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:07.148932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"]
Sep 10 16:52:07.151248 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:52:07.151221    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f9e6567_4107_49d3_b9ab_1fc9449c9a61.slice/crio-f53bc23d85b716a293fe197dd839a57054e9a94c7c180b779f6c25d48e8113d2 WatchSource:0}: Error finding container f53bc23d85b716a293fe197dd839a57054e9a94c7c180b779f6c25d48e8113d2: Status 404 returned error can't find the container with id f53bc23d85b716a293fe197dd839a57054e9a94c7c180b779f6c25d48e8113d2
Sep 10 16:52:07.762507 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:07.762463    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55" event={"ID":"4f9e6567-4107-49d3-b9ab-1fc9449c9a61","Type":"ContainerStarted","Data":"3f2103ba30d02841948cd2a1e352abdaa20d1c24d7bca8d2f4d3f8782708bf56"}
Sep 10 16:52:07.762893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:07.762528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:07.762893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:07.762545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55" event={"ID":"4f9e6567-4107-49d3-b9ab-1fc9449c9a61","Type":"ContainerStarted","Data":"f53bc23d85b716a293fe197dd839a57054e9a94c7c180b779f6c25d48e8113d2"}
Sep 10 16:52:07.789299 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:07.789224    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55" podStartSLOduration=1.309714346 podStartE2EDuration="1.789207502s" totalImagesPullingTime="479.493156ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:52:07.785219222 +0000 UTC m=+2100.187020348" watchObservedRunningTime="2026-09-10 16:52:07.789207502 +0000 UTC m=+2100.191008674"
Sep 10 16:52:08.304942 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:08.304913    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:52:08.308929 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:08.308905    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:52:13.775083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:13.775051    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ddc7f4579-pwv55"
Sep 10 16:52:23.628993 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:23.628948    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:52:23.629455 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:23.629215    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" podUID="319286b9-4ff5-4560-98bf-6fc2d8630862" containerName="maas-api" containerID="cri-o://7ad7884c9508c35da99017e99cc309805f274446832ac096ddec11ca69f11335" gracePeriod=30
Sep 10 16:52:24.870179 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.870144    2577 generic.go:356] "Generic (PLEG): container finished" podID="319286b9-4ff5-4560-98bf-6fc2d8630862" containerID="7ad7884c9508c35da99017e99cc309805f274446832ac096ddec11ca69f11335" exitCode=0
Sep 10 16:52:24.870539 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.870217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" event={"ID":"319286b9-4ff5-4560-98bf-6fc2d8630862","Type":"ContainerDied","Data":"7ad7884c9508c35da99017e99cc309805f274446832ac096ddec11ca69f11335"}
Sep 10 16:52:24.886734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.886712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:52:24.996461 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.996365    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") pod \"319286b9-4ff5-4560-98bf-6fc2d8630862\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") "
Sep 10 16:52:24.996461 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.996438    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca\") pod \"319286b9-4ff5-4560-98bf-6fc2d8630862\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") "
Sep 10 16:52:24.996705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.996502    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls\") pod \"319286b9-4ff5-4560-98bf-6fc2d8630862\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") "
Sep 10 16:52:24.996705 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.996548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhh8c\" (UniqueName: \"kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c\") pod \"319286b9-4ff5-4560-98bf-6fc2d8630862\" (UID: \"319286b9-4ff5-4560-98bf-6fc2d8630862\") "
Sep 10 16:52:24.996880 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.996846    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "319286b9-4ff5-4560-98bf-6fc2d8630862" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 16:52:24.998678 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.998642    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "319286b9-4ff5-4560-98bf-6fc2d8630862" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:52:24.998805 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.998789    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c" (OuterVolumeSpecName: "kube-api-access-qhh8c") pod "319286b9-4ff5-4560-98bf-6fc2d8630862" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862"). InnerVolumeSpecName "kube-api-access-qhh8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 16:52:24.998939 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:24.998916    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "319286b9-4ff5-4560-98bf-6fc2d8630862" (UID: "319286b9-4ff5-4560-98bf-6fc2d8630862"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 16:52:25.097780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.097738    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/319286b9-4ff5-4560-98bf-6fc2d8630862-openshift-service-ca\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:52:25.097780 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.097779    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-maas-api-tls\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:52:25.097983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.097793    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhh8c\" (UniqueName: \"kubernetes.io/projected/319286b9-4ff5-4560-98bf-6fc2d8630862-kube-api-access-qhh8c\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:52:25.097983 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.097806    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/319286b9-4ff5-4560-98bf-6fc2d8630862-metrics-certs\") on node \"ip-10-0-136-40.ec2.internal\" DevicePath \"\""
Sep 10 16:52:25.878042 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.878011    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"
Sep 10 16:52:25.878498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.878048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t" event={"ID":"319286b9-4ff5-4560-98bf-6fc2d8630862","Type":"ContainerDied","Data":"e60ee81e765b48024c9e5959c77347535ca1e71eeae03c089d5648a64ad8e717"}
Sep 10 16:52:25.878498 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.878110    2577 scope.go:117] "RemoveContainer" containerID="7ad7884c9508c35da99017e99cc309805f274446832ac096ddec11ca69f11335"
Sep 10 16:52:25.902675 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.902651    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:52:25.904887 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:25.904867    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6c68568df5-f8x2t"]
Sep 10 16:52:26.113222 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:26.113183    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="319286b9-4ff5-4560-98bf-6fc2d8630862" path="/var/lib/kubelet/pods/319286b9-4ff5-4560-98bf-6fc2d8630862/volumes"
Sep 10 16:52:41.653251 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:41.653218    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c6rq9_ac4202d6-b576-4a49-80d9-17499f64277a/manager/0.log"
Sep 10 16:52:41.772025 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:41.771987    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mz45n_cb4182c8-308d-4eec-b0f4-b25f3a998ca0/manager/0.log"
Sep 10 16:52:41.887107 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:41.887075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bb45c94d6-xtvmv_467233d7-5c3e-48c1-9a6c-d5243ec980ea/manager/0.log"
Sep 10 16:52:42.471610 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:42.471579    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b77968d9-qr82g_350e4f97-8b25-4e12-87cb-76f4da7476aa/copy-manifests/0.log"
Sep 10 16:52:42.480958 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:42.480933    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b77968d9-qr82g_350e4f97-8b25-4e12-87cb-76f4da7476aa/manager/0.log"
Sep 10 16:52:42.922432 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:42.922403    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ddc7f4579-pwv55_4f9e6567-4107-49d3-b9ab-1fc9449c9a61/maas-api/0.log"
Sep 10 16:52:43.246894 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:43.246821    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-hhvqk_538982de-b548-4af5-a889-27ad60adf5cf/postgres/0.log"
Sep 10 16:52:44.101286 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:44.101244    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tprdg_50cd86ce-43c2-484e-92a3-2b746f933843/manager/0.log"
Sep 10 16:52:44.215104 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:44.215075    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hqjz4_fdaeb9a9-0402-4834-a07e-bb0352992943/manager/0.log"
Sep 10 16:52:44.322074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:44.322039    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8jh4k_a4d06266-939a-46d8-8561-26d17e24b558/kuadrant-console-plugin/0.log"
Sep 10 16:52:44.435495 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:44.435461    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-crzrw_27888c2d-aa62-43ab-b234-7398d2ace5ec/manager/0.log"
Sep 10 16:52:44.658893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:44.658857    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-tsxwj_9a2f4b97-7fb7-4b92-ac37-7666a5f774f4/manager/0.log"
Sep 10 16:52:45.236402 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:45.236372    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cfcc4b4c4-5fzmm_8a6f63ec-9e5a-4c75-af93-00e618ec9a51/kube-auth-proxy/0.log"
Sep 10 16:52:45.580709 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:45.580629    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k2fkh_8a1a84f0-74ab-417c-ad76-a66f38857ee5/payload-pre-processing/0.log"
Sep 10 16:52:45.806397 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:45.806365    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59648f9df8-wbvrp_0094cd2c-52d0-4501-be7d-cb3b6b13b573/router/0.log"
Sep 10 16:52:46.253181 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:46.253095    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rzj5d_1f745e6b-7bfc-4055-8f04-0cb94023e435/main/0.log"
Sep 10 16:52:46.568838 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:46.568754    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fsxqq9_d6015082-18e4-4b0a-b776-5d53164e8b17/main/0.log"
Sep 10 16:52:52.102166 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.102125    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbsrw/must-gather-lljfs"]
Sep 10 16:52:52.102729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.102550    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="319286b9-4ff5-4560-98bf-6fc2d8630862" containerName="maas-api"
Sep 10 16:52:52.102729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.102562    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="319286b9-4ff5-4560-98bf-6fc2d8630862" containerName="maas-api"
Sep 10 16:52:52.102729 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.102641    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="319286b9-4ff5-4560-98bf-6fc2d8630862" containerName="maas-api"
Sep 10 16:52:52.105844 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.105826    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.109305 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.109252    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbsrw\"/\"kube-root-ca.crt\""
Sep 10 16:52:52.109305 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.109293    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rbsrw\"/\"openshift-service-ca.crt\""
Sep 10 16:52:52.109523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.109293    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rbsrw\"/\"default-dockercfg-8nc6g\""
Sep 10 16:52:52.124808 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.124771    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbsrw/must-gather-lljfs"]
Sep 10 16:52:52.244309 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.244244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jbl6\" (UniqueName: \"kubernetes.io/projected/ac025aac-80d0-4099-9552-a000c5139d00-kube-api-access-8jbl6\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.244477 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.244390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac025aac-80d0-4099-9552-a000c5139d00-must-gather-output\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.345751 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.345714    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac025aac-80d0-4099-9552-a000c5139d00-must-gather-output\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.345932 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.345776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jbl6\" (UniqueName: \"kubernetes.io/projected/ac025aac-80d0-4099-9552-a000c5139d00-kube-api-access-8jbl6\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.346084 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.346062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ac025aac-80d0-4099-9552-a000c5139d00-must-gather-output\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.354975 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.354907    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jbl6\" (UniqueName: \"kubernetes.io/projected/ac025aac-80d0-4099-9552-a000c5139d00-kube-api-access-8jbl6\") pod \"must-gather-lljfs\" (UID: \"ac025aac-80d0-4099-9552-a000c5139d00\") " pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.423256 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.423220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbsrw/must-gather-lljfs"
Sep 10 16:52:52.695474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.695449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbsrw/must-gather-lljfs"]
Sep 10 16:52:52.697367 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:52:52.697339    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podac025aac_80d0_4099_9552_a000c5139d00.slice/crio-f3986132edcbe0606cf53d65a73a4d4ba57fff1555509332b51f59e7014866a8 WatchSource:0}: Error finding container f3986132edcbe0606cf53d65a73a4d4ba57fff1555509332b51f59e7014866a8: Status 404 returned error can't find the container with id f3986132edcbe0606cf53d65a73a4d4ba57fff1555509332b51f59e7014866a8
Sep 10 16:52:52.699613 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:52.699592    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 16:52:53.037886 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:53.037799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbsrw/must-gather-lljfs" event={"ID":"ac025aac-80d0-4099-9552-a000c5139d00","Type":"ContainerStarted","Data":"f3986132edcbe0606cf53d65a73a4d4ba57fff1555509332b51f59e7014866a8"}
Sep 10 16:52:54.050160 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:54.049181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbsrw/must-gather-lljfs" event={"ID":"ac025aac-80d0-4099-9552-a000c5139d00","Type":"ContainerStarted","Data":"f2d22360b461af095b79416c89a5653f321cb20431efa1eac674edb81d46d844"}
Sep 10 16:52:54.050160 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:54.049241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbsrw/must-gather-lljfs" event={"ID":"ac025aac-80d0-4099-9552-a000c5139d00","Type":"ContainerStarted","Data":"f6a5431a4882603a30078c1f3354ceaf05f882a671230c6065231f5e36d63621"}
Sep 10 16:52:54.069068 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:54.069006    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rbsrw/must-gather-lljfs" podStartSLOduration=1.150958494 podStartE2EDuration="2.068983799s" totalImagesPullingTime="918.025305ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 16:52:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:52:54.068264359 +0000 UTC m=+2146.470065487" watchObservedRunningTime="2026-09-10 16:52:54.068983799 +0000 UTC m=+2146.470784929"
Sep 10 16:52:55.282804 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:55.282770    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xwrj7_0bdb4c52-c0b9-4029-a2b2-ed39461af0ce/global-pull-secret-syncer/0.log"
Sep 10 16:52:55.393023 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:55.392987    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w27lk_72045530-1543-405a-9763-852fb5e9c22f/konnectivity-agent/0.log"
Sep 10 16:52:55.415684 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:52:55.415654    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-40.ec2.internal_251c9980d3ffefc3102b5100c4ca2cb5/haproxy/0.log"
Sep 10 16:53:00.253561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:00.253461    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-tprdg_50cd86ce-43c2-484e-92a3-2b746f933843/manager/0.log"
Sep 10 16:53:00.286082 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:00.286054    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-hqjz4_fdaeb9a9-0402-4834-a07e-bb0352992943/manager/0.log"
Sep 10 16:53:00.315677 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:00.315649    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-8jh4k_a4d06266-939a-46d8-8561-26d17e24b558/kuadrant-console-plugin/0.log"
Sep 10 16:53:00.359523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:00.359493    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-crzrw_27888c2d-aa62-43ab-b234-7398d2ace5ec/manager/0.log"
Sep 10 16:53:00.458873 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:00.458842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-tsxwj_9a2f4b97-7fb7-4b92-ac37-7666a5f774f4/manager/0.log"
Sep 10 16:53:02.175580 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:02.175548    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-5z5wk_a42b4b80-9ace-4a5f-b0e0-425e2b1c757a/cluster-monitoring-operator/0.log"
Sep 10 16:53:02.448622 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:02.448538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2zpl_f53b5235-c8fe-4048-9f56-3e7d69644030/node-exporter/0.log"
Sep 10 16:53:02.483026 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:02.482995    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2zpl_f53b5235-c8fe-4048-9f56-3e7d69644030/kube-rbac-proxy/0.log"
Sep 10 16:53:02.532965 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:02.532933    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s2zpl_f53b5235-c8fe-4048-9f56-3e7d69644030/init-textfile/0.log"
Sep 10 16:53:05.239474 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.239444    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/1.log"
Sep 10 16:53:05.247623 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.247597    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cpn8t_a7d7b92b-7659-4bf4-a95a-4d5364e3578a/console-operator/2.log"
Sep 10 16:53:05.644816 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.644766    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"]
Sep 10 16:53:05.653734 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.653701    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.660430 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.660402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"]
Sep 10 16:53:05.692524 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.692482    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-podres\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.692821 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.692778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkgpw\" (UniqueName: \"kubernetes.io/projected/fc9b7de3-3056-4845-af10-90b150834bac-kube-api-access-xkgpw\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.692935 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.692834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-proc\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.693008 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.692932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-lib-modules\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.693064 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.693013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-sys\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794074 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794030    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-podres\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794097    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkgpw\" (UniqueName: \"kubernetes.io/projected/fc9b7de3-3056-4845-af10-90b150834bac-kube-api-access-xkgpw\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-proc\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-lib-modules\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794270 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794248    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-sys\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-sys\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794561 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794461    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-proc\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794681 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-lib-modules\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.794681 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.794605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fc9b7de3-3056-4845-af10-90b150834bac-podres\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.804907 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.804873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkgpw\" (UniqueName: \"kubernetes.io/projected/fc9b7de3-3056-4845-af10-90b150834bac-kube-api-access-xkgpw\") pod \"perf-node-gather-daemonset-xrgkc\" (UID: \"fc9b7de3-3056-4845-af10-90b150834bac\") " pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:05.838855 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.838826    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-78956c4984-l4qmm_97654c5d-1dbf-42e9-bbbe-d5fa52cff4a1/console/0.log"
Sep 10 16:53:05.877673 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.877644    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-j95rf_d73dbe40-0057-41ad-a3fa-c6e668e82709/download-server/0.log"
Sep 10 16:53:05.968464 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:05.968353    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:06.276523 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:06.275469    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"]
Sep 10 16:53:06.278446 ip-10-0-136-40 kubenswrapper[2577]: W0910 16:53:06.278417    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfc9b7de3_3056_4845_af10_90b150834bac.slice/crio-ee8c4d7fa5ced68e9fa83cb05e42f916152eac4a63751c561a37a44e0c2c9419 WatchSource:0}: Error finding container ee8c4d7fa5ced68e9fa83cb05e42f916152eac4a63751c561a37a44e0c2c9419: Status 404 returned error can't find the container with id ee8c4d7fa5ced68e9fa83cb05e42f916152eac4a63751c561a37a44e0c2c9419
Sep 10 16:53:06.571776 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:06.571652    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-fd8s7_29da8dc0-47a0-4dfa-ac9e-82807f94fefc/volume-data-source-validator/0.log"
Sep 10 16:53:07.187895 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.187851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc" event={"ID":"fc9b7de3-3056-4845-af10-90b150834bac","Type":"ContainerStarted","Data":"bd1d07294f62b3e2b5ef0bfaa161956620ce3a85da05ee1f1db2e736c0261b42"}
Sep 10 16:53:07.188099 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.187929    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc" event={"ID":"fc9b7de3-3056-4845-af10-90b150834bac","Type":"ContainerStarted","Data":"ee8c4d7fa5ced68e9fa83cb05e42f916152eac4a63751c561a37a44e0c2c9419"}
Sep 10 16:53:07.188773 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.188744    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:07.206142 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.206094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc" podStartSLOduration=2.206080279 podStartE2EDuration="2.206080279s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 16:53:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 16:53:07.205748646 +0000 UTC m=+2159.607549774" watchObservedRunningTime="2026-09-10 16:53:07.206080279 +0000 UTC m=+2159.607881406"
Sep 10 16:53:07.764907 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.764877    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm6h9_3f6d7871-16e9-416f-b9ca-6239be4b7fa4/dns/0.log"
Sep 10 16:53:07.787364 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.787331    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-tm6h9_3f6d7871-16e9-416f-b9ca-6239be4b7fa4/kube-rbac-proxy/0.log"
Sep 10 16:53:07.860064 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:07.860036    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-r84rk_9ab87d95-1588-4a35-9211-d0d53b937dd2/dns-node-resolver/0.log"
Sep 10 16:53:08.426893 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:08.426865    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d24lj_1500121e-5856-4e8a-96a1-a7815dc5905c/node-ca/0.log"
Sep 10 16:53:09.747788 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:09.747761    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cfcc4b4c4-5fzmm_8a6f63ec-9e5a-4c75-af93-00e618ec9a51/kube-auth-proxy/0.log"
Sep 10 16:53:09.848970 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:09.848942    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k2fkh_8a1a84f0-74ab-417c-ad76-a66f38857ee5/payload-pre-processing/0.log"
Sep 10 16:53:09.915354 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:09.915324    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59648f9df8-wbvrp_0094cd2c-52d0-4501-be7d-cb3b6b13b573/router/0.log"
Sep 10 16:53:10.874266 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:10.874237    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xm69q_9fc228f9-6b63-4f2b-aca1-c4720725d2d0/serve-healthcheck-canary/0.log"
Sep 10 16:53:11.319583 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:11.319503    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ctsf5_71fd4ecd-9ead-4a5f-8827-82fbd67d4812/insights-operator/0.log"
Sep 10 16:53:11.320058 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:11.320039    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-ctsf5_71fd4ecd-9ead-4a5f-8827-82fbd67d4812/insights-operator/1.log"
Sep 10 16:53:11.495091 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:11.495065    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9zm4_195d72b1-2a60-4b7b-8591-2a4026d970ec/kube-rbac-proxy/0.log"
Sep 10 16:53:11.517763 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:11.517739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9zm4_195d72b1-2a60-4b7b-8591-2a4026d970ec/exporter/0.log"
Sep 10 16:53:11.540018 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:11.539991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q9zm4_195d72b1-2a60-4b7b-8591-2a4026d970ec/extractor/0.log"
Sep 10 16:53:13.706852 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:13.706822    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c6rq9_ac4202d6-b576-4a49-80d9-17499f64277a/manager/0.log"
Sep 10 16:53:13.929908 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:13.929876    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mz45n_cb4182c8-308d-4eec-b0f4-b25f3a998ca0/manager/0.log"
Sep 10 16:53:13.983965 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:13.983877    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7bb45c94d6-xtvmv_467233d7-5c3e-48c1-9a6c-d5243ec980ea/manager/0.log"
Sep 10 16:53:14.223259 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:14.223201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rbsrw/perf-node-gather-daemonset-xrgkc"
Sep 10 16:53:14.282531 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:14.282450    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b77968d9-qr82g_350e4f97-8b25-4e12-87cb-76f4da7476aa/manager/0.log"
Sep 10 16:53:14.305001 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:14.304973    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57b77968d9-qr82g_350e4f97-8b25-4e12-87cb-76f4da7476aa/copy-manifests/0.log"
Sep 10 16:53:16.244241 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:16.244209    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cfcc4b4c4-5fzmm_8a6f63ec-9e5a-4c75-af93-00e618ec9a51/kube-auth-proxy/0.log"
Sep 10 16:53:16.319984 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:16.319901    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k2fkh_8a1a84f0-74ab-417c-ad76-a66f38857ee5/payload-pre-processing/0.log"
Sep 10 16:53:16.362223 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:16.362191    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59648f9df8-wbvrp_0094cd2c-52d0-4501-be7d-cb3b6b13b573/router/0.log"
Sep 10 16:53:18.264323 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:18.264295    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5cfcc4b4c4-5fzmm_8a6f63ec-9e5a-4c75-af93-00e618ec9a51/kube-auth-proxy/0.log"
Sep 10 16:53:18.346231 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:18.346192    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-k2fkh_8a1a84f0-74ab-417c-ad76-a66f38857ee5/payload-pre-processing/0.log"
Sep 10 16:53:18.386652 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:18.386619    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-59648f9df8-wbvrp_0094cd2c-52d0-4501-be7d-cb3b6b13b573/router/0.log"
Sep 10 16:53:21.032849 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:21.032820    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-bnx5n_e543fad1-4ed6-40f0-bdc7-f084a19fe3bf/migrator/0.log"
Sep 10 16:53:21.054202 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:21.054177    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-bnx5n_e543fad1-4ed6-40f0-bdc7-f084a19fe3bf/graceful-termination/0.log"
Sep 10 16:53:21.412372 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:21.412336    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-hln7r_0d2e7938-d60f-4b3f-9d05-562219253f14/kube-storage-version-migrator-operator/1.log"
Sep 10 16:53:21.413217 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:21.413195    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-hln7r_0d2e7938-d60f-4b3f-9d05-562219253f14/kube-storage-version-migrator-operator/0.log"
Sep 10 16:53:22.509216 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.509183    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-95whm_dc1d39a1-3c08-48dd-84b3-ff9ebbf63831/kube-multus/0.log"
Sep 10 16:53:22.531920 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.531886    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/kube-multus-additional-cni-plugins/0.log"
Sep 10 16:53:22.552292 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.552255    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/egress-router-binary-copy/0.log"
Sep 10 16:53:22.573083 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.573053    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/cni-plugins/0.log"
Sep 10 16:53:22.596127 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.596101    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/bond-cni-plugin/0.log"
Sep 10 16:53:22.617463 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.617428    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/routeoverride-cni/0.log"
Sep 10 16:53:22.638333 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.638306    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/whereabouts-cni-bincopy/0.log"
Sep 10 16:53:22.659850 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:22.659810    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hr6l7_534d15f4-9147-439b-8a20-9d951e5cf6f3/whereabouts-cni/0.log"
Sep 10 16:53:23.035111 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:23.035078    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87mpg_13542d90-fc55-4433-aeb2-8ebf3cc57ee0/network-metrics-daemon/0.log"
Sep 10 16:53:23.052692 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:23.052668    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-87mpg_13542d90-fc55-4433-aeb2-8ebf3cc57ee0/kube-rbac-proxy/0.log"
Sep 10 16:53:24.377551 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.377511    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/ovn-controller/0.log"
Sep 10 16:53:24.393487 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.393458    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/ovn-acl-logging/0.log"
Sep 10 16:53:24.419336 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.419304    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/kube-rbac-proxy-node/0.log"
Sep 10 16:53:24.443579 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.443550    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 16:53:24.469221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.469173    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/northd/0.log"
Sep 10 16:53:24.493308 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.493254    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/nbdb/0.log"
Sep 10 16:53:24.516760 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.516734    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/sbdb/0.log"
Sep 10 16:53:24.590221 ip-10-0-136-40 kubenswrapper[2577]: I0910 16:53:24.590186    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-n746n_89b9e2a7-13a7-40eb-9194-06c2f9bad9a2/ovnkube-controller/0.log"