Jul 08 03:17:45.745841 ip-10-0-137-160 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 08 03:17:45.745852 ip-10-0-137-160 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 08 03:17:45.745862 ip-10-0-137-160 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 08 03:17:45.746180 ip-10-0-137-160 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 08 03:17:55.891734 ip-10-0-137-160 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 08 03:17:55.891750 ip-10-0-137-160 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot a220196395d04eb18429fc60c4e4b89c --
Jul 08 03:19:48.781414 ip-10-0-137-160 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 03:19:49.163006 ip-10-0-137-160 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:49.163006 ip-10-0-137-160 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 03:19:49.163006 ip-10-0-137-160 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:49.163006 ip-10-0-137-160 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 03:19:49.163006 ip-10-0-137-160 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 03:19:49.163722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.163638    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 03:19:49.169363 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169341    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.169363 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169361    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.169363 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169365    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169368    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169371    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169374    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169376    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169382    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169386    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169397    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169400    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169403    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169405    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169408    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169410    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169412    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169415    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169418    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169420    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169423    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169425    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169428    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.169459 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169431    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169434    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169436    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169438    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169441    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169443    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169446    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169448    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169451    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169454    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169456    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169459    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169462    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169464    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169468    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169471    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169474    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169476    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169479    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169482    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.170081 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169484    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169487    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169489    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169491    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169494    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169496    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169499    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169501    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169504    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169506    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169509    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169511    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169513    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169516    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169519    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169522    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169524    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169527    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169529    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.170918 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169534    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169537    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169540    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169543    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169545    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169548    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169551    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169553    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169555    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169558    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169560    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169563    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169565    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169568    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169570    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169575    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169578    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169581    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169583    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169586    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.171473 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169588    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169591    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169593    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169596    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169599    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169601    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169603    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169607    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169609    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169613    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169615    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169617    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169620    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169623    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169625    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169628    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169630    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.169633    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171422    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.171955 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171434    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171438    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171442    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171445    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171448    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171451    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171455    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171458    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171460    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171463    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171466    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171469    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171471    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171474    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171477    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171480    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171483    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171487    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171491    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.172423 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171494    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171497    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171499    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171502    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171505    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171508    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171511    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171513    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171520    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171523    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171526    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171528    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171531    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171536    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171538    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171541    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171543    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171546    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171549    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171551    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.172950 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171554    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171556    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171558    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171561    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171563    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171566    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171568    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171571    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171574    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171576    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171578    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171581    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171584    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171586    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171589    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171591    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171593    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171596    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171598    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171600    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.173463 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171604    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171608    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171612    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171615    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171618    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171621    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171624    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171626    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171629    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171631    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171634    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171636    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171638    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171641    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171643    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171645    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171648    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171650    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171652    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171654    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.173954 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171657    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171659    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171662    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171665    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171667    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171669    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171672    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171674    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171678    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171682    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171684    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171687    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171689    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171691    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171694    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171697    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171699    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171702    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.171705    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.174446 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172080    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172084    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172087    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172090    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172092    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172095    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172098    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172100    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172103    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172105    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172108    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172112    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172115    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172118    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172121    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172124    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172127    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172130    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172133    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.174890 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172151    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172154    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172157    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172159    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172162    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172165    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172167    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172169    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172172    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172175    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172177    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172180    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172182    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172185    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172188    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172191    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172193    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172196    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172198    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172201    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.175364 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172203    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172205    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172208    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172210    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172213    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172216    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172218    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172221    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172223    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172226    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172229    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172231    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172234    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172236    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172238    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172241    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172243    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172245    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172248    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.175830 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172252    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172255    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172258    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172261    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172264    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172267    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172269    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172272    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172275    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172277    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172279    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172282    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172284    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172286    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172295    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172297    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172300    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172302    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172305    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.176325 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172307    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172311    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172314    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172316    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172318    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172321    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172324    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172326    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172329    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172331    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172333    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172336    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172338    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172341    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172343    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172346    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172349    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172352    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172354    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172357    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.176838 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172359    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.172362    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172431    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172438    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172445    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172449    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172453    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172456    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172460    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172464    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172467    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172470    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172473    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172476    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172479    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172482    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172485    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172487    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172491    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172493    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172497    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172500    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172503    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172506    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 03:19:49.177355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172509    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172513    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172516    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172519    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172525    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172528    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172531    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172533    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172536    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172539    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172543    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172546    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172549    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172551    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172554    2563 flags.go:64] FLAG: --enable-server="true"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172557    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172562    2563 flags.go:64] FLAG: --event-burst="100"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172565    2563 flags.go:64] FLAG: --event-qps="50"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172567    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172570    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172573    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172576    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172579    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172582    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172585    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 08 03:19:49.177910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172587    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172590    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172594    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172596    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172602    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172605    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172608    2563 flags.go:64] FLAG: --feature-gates=""
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172611    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172614    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172617    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172620    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172624    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172627    2563 flags.go:64] FLAG: --help="false"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172630    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172633    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172636    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172639    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172643    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172646    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172649    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172651    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172654    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172656    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 03:19:49.178562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172659    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172662    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172665    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172667    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172670    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172673    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172675    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172678    2563 flags.go:64] FLAG: --lock-file=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172680    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172683    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172686    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172696    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172699    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172703    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172705    2563 flags.go:64] FLAG: --logging-format="text"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172708    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172712    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172714    2563 flags.go:64] FLAG: --manifest-url=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172717    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172721    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172725    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172729    2563 flags.go:64] FLAG: --max-pods="110"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172732    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172735    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172737    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 03:19:49.179189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172740    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172743    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172746    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172749    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172755    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172758    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172761    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172764    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172767    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172773    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172775    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172778    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172781    2563 flags.go:64] FLAG: --port="10250"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172784    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172786    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06e50e73704a0d7a4"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172789    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172792    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172795    2563 flags.go:64] FLAG: --register-node="true"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172798    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172801    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172806    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172810    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172813    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172816    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172819    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 03:19:49.179798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172822    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172825    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172828    2563 flags.go:64] FLAG: --runonce="false"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172831    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172834    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172837    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172840    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172842    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172845    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172848    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172850    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172853    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172856    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172859    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172861    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172864    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172867    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172869    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172874    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172877    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172879    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172883    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172886    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172888    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172890    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 03:19:49.180424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172893    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172896    2563 flags.go:64] FLAG: --v="2"
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172900    2563 flags.go:64] FLAG: --version="false"
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172906    2563 flags.go:64] FLAG: --vmodule=""
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172909    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.172913    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173033    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173037    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173040    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173044    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173046    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173049    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173052    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173055    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173057    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173060    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173063    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173066    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173068    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173080    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173082    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.181013 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173085    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173088    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173090    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173093    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173095    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173097    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173100    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173102    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173105    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173107    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173110    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173112    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173114    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173117    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173120    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173124    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173126    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173129    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173131    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.181553 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173149    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173154    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173156    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173159    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173162    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173165    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173167    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173170    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173174    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173177    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173180    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173183    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173186    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173188    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173191    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173194    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173196    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173200    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173203    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.182049 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173205    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173209    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173211    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173214    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173216    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173219    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173221    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173223    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173228    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173230    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173233    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173236    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173238    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173240    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173243    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173246    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173248    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173250    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173253    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173255    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.182528 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173257    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173260    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173262    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173265    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173267    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173269    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173272    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173274    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173276    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173279    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173281    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173283    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173286    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173288    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173291    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173293    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173295    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173298    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173300    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173302    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.183005 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173306    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173309    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173311    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173314    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173317    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.173319    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.173328    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180275    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180292    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180305    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180310    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180364    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180369    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180373    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.183561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180375    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180378    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180381    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180384    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180386    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180389    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180392    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180394    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180397    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180399    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180403    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180405    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180408    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180411    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180413    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180416    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180418    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180421    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180423    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180426    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.183913 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180428    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180432    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180434    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180437    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180440    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180442    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180445    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180448    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180450    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180453    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180455    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180458    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180460    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180463    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180465    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180468    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180470    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180473    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180475    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.184415 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180479    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180483    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180486    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180489    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180492    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180495    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180499    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180501    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180505    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180509    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180512    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180515    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180518    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180521    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180523    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180526    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180529    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180532    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180534    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.184896 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180537    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180540    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180542    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180545    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180547    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180550    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180552    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180554    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180557    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180560    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180562    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180565    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180567    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180570    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180572    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180575    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180577    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180580    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180582    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180585    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.185376 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180587    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180589    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180592    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180594    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180597    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180599    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180602    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180604    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180606    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180609    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180611    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180614    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180616    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180619    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180623    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180627    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180629    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.185868 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180631    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180636    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180741    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180746    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180749    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180751    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180754    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180756    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180759    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180762    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180764    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180767    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180769    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180772    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180774    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 03:19:49.186311 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180776    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180779    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180781    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180784    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180786    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180789    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180791    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180793    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180796    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180798    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180801    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180804    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180806    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180809    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180811    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180814    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180816    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180819    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180821    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180824    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 03:19:49.186674 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180826    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180828    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180831    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180833    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180835    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180838    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180840    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180842    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180845    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180847    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180850    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180853    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180855    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180857    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180860    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180864    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180867    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180869    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180872    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 03:19:49.187240 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180874    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180876    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180879    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180881    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180884    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180887    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180890    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180892    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180895    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180897    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180899    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180902    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180904    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180907    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180909    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180913    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180916    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180918    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180921    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 03:19:49.187723 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180924    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180926    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180929    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180932    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180934    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180937    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180939    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180942    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180946    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180949    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180952    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180954    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180957    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180959    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180962    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180965    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180967    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180969    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180972    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 03:19:49.188206 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180975    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180978    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180980    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180983    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180985    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180987    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180990    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180992    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:49.180994    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.180999    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.181117    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.183224    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.184466    2563 server.go:1019] "Starting client certificate rotation"
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.184573    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 03:19:49.188694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.184634    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 03:19:49.189032 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.187419    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 03:19:49.189032 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.188926    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 03:19:49.195347 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.195329    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 08 03:19:49.197983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.197966    2563 log.go:25] "Validated CRI v1 image API"
Jul 08 03:19:49.199621 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.199604    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 03:19:49.201908 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.201887    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 7fc53fd4-b870-4787-a61c-8f8b6f6f22bb:/dev/nvme0n1p4 dedf7e9f-e8ff-4d38-bb24-e89542ada3fb:/dev/nvme0n1p3]
Jul 08 03:19:49.201988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.201908    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 03:19:49.207577 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.207460    2563 manager.go:216] Machine: {Timestamp:2026-07-08 03:19:49.205989921 +0000 UTC m=+0.314335852 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101475 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2174809634729f67a2495622eaced3 SystemUUID:ec217480-9634-729f-67a2-495622eaced3 BootID:a2201963-95d0-4eb1-8429-fc60c4e4b89c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c6:bd:53:b7:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c6:bd:53:b7:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:b6:9c:e7:e7:0a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 03:19:49.207577 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.207573    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 03:19:49.207690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.207660    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 03:19:49.210694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.210666    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 03:19:49.210837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.210699    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-160.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 03:19:49.211443 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.211432    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 03:19:49.211483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.211446    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 03:19:49.211483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.211453    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 03:19:49.212071 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.212061    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 03:19:49.213396 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.213367    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 03:19:49.213454 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.213420    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 03:19:49.216077 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216065    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 03:19:49.216262 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216251    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 03:19:49.216303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216283    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 03:19:49.216303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216297    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 03:19:49.216358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216311    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 03:19:49.216358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216320    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 08 03:19:49.216358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216332    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 03:19:49.216713 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216683    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 03:19:49.216713 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.216706    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 03:19:49.218752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.218737    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 03:19:49.220512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.220499    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 03:19:49.220549 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.220529    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 03:19:49.221654 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221642    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 03:19:49.221695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221664    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 03:19:49.221695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221670    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 03:19:49.221695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221676    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 03:19:49.221695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221686    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 03:19:49.221695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221692    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221702    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221707    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221714    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221722    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221743    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 03:19:49.221815 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.221757    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 03:19:49.222688 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.222679    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 03:19:49.222718 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.222688    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 03:19:49.226282 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.226268    2563 server.go:1268] "Started kubelet"
Jul 08 03:19:49.226377 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.226352    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 03:19:49.226433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.226353    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 03:19:49.226491 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.226470    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 03:19:49.226876 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.226855    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-160.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 03:19:49.227254 ip-10-0-137-160 systemd[1]: Started Kubernetes Kubelet.
Jul 08 03:19:49.230027 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.229960    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 03:19:49.230153 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.230041    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 03:19:49.230802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.230632    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 03:19:49.231974 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.231948    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 03:19:49.232589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.232574    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 03:19:49.234124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.234102    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v9vfh"
Jul 08 03:19:49.236218 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.235086    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-160.ec2.internal.18c03254199046c5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-160.ec2.internal,UID:ip-10-0-137-160.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-160.ec2.internal,},FirstTimestamp:2026-07-08 03:19:49.226243781 +0000 UTC m=+0.334589713,LastTimestamp:2026-07-08 03:19:49.226243781 +0000 UTC m=+0.334589713,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-160.ec2.internal,}"
Jul 08 03:19:49.237504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.237488    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 03:19:49.237575 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.237507    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 03:19:49.237630 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.237604    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.238117 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238099    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 03:19:49.238117 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238101    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 03:19:49.238261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238129    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 03:19:49.238686 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238566    2563 factory.go:55] Registering systemd factory
Jul 08 03:19:49.238686 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238580    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 08 03:19:49.238939 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238924    2563 factory.go:153] Registering CRI-O factory
Jul 08 03:19:49.239002 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238942    2563 factory.go:223] Registration of the crio container factory successfully
Jul 08 03:19:49.239002 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238991    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 03:19:49.239116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.239015    2563 factory.go:103] Registering Raw factory
Jul 08 03:19:49.239116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.238991    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 03:19:49.239116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.239030    2563 manager.go:1196] Started watching for new ooms in manager
Jul 08 03:19:49.239116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.239053    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 03:19:49.239448 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.239435    2563 manager.go:349] Starting recovery of all containers
Jul 08 03:19:49.242321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.242299    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v9vfh"
Jul 08 03:19:49.248802 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.248632    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 03:19:49.248802 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.248733    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-160.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 03:19:49.251358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.251342    2563 manager.go:354] Recovery completed
Jul 08 03:19:49.255678 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.255664    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.257722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.257705    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.257802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.257734    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.257802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.257744    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.258307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258292    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 03:19:49.258307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258304    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 03:19:49.258385 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258312    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.258385 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258324    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 03:19:49.258988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258970    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.259055 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.258998    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.259055 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.259016    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.259828 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.259817    2563 policy_none.go:49] "None policy: Start"
Jul 08 03:19:49.259878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.259832    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 03:19:49.259878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.259842    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 03:19:49.261184 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.261171    2563 policy_none.go:47] "Start"
Jul 08 03:19:49.302701 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.302684    2563 manager.go:337] "Starting Device Plugin manager"
Jul 08 03:19:49.302783 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.302726    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 03:19:49.302783 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.302739    2563 server.go:87] "Starting device plugin registration server"
Jul 08 03:19:49.303011 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.302999    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 03:19:49.303056 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.303015    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 03:19:49.303112 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.303098    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 03:19:49.303479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.303460    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 03:19:49.303837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.303817    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 03:19:49.305454 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.305434    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 03:19:49.305516 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.305487    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.376966 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.376936    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 03:19:49.378242 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.378217    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 03:19:49.378324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.378258    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 03:19:49.378324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.378282    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 03:19:49.378324 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.378315    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 03:19:49.380910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.380889    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:49.404093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.404071    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.405003 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.404985    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.405086 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.405018    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.405086 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.405032    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.405086 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.405054    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.411644 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.411631    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.411693 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.411652    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-160.ec2.internal\": node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.424759 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.424713    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.478849 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.478825    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"]
Jul 08 03:19:49.478926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.478897    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.479750 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.479733    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.479823 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.479763    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.479823 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.479774    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.481804 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.481792    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.481922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.481908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.481965 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.481940    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.482464 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482442    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.482464 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482464    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.482575 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482473    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.482575 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482524    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.482575 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482543    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.482575 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.482555    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.483960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.483945    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.484005 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.483973    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 03:19:49.485262 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.485247    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 03:19:49.485322 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.485273    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 03:19:49.485322 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.485284    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeHasSufficientPID"
Jul 08 03:19:49.513847 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.513826    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-160.ec2.internal\" not found" node="ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.518110 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.518094    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-160.ec2.internal\" not found" node="ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.525148 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.525122    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.540461 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.540440    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.540537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.540464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.540537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.540482    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c4d2eab85087dd7f7d7ed97e5b0c0e8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-160.ec2.internal\" (UID: \"4c4d2eab85087dd7f7d7ed97e5b0c0e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.626230 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.626192    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.641634 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.641735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.641735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c4d2eab85087dd7f7d7ed97e5b0c0e8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-160.ec2.internal\" (UID: \"4c4d2eab85087dd7f7d7ed97e5b0c0e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.641735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641703    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.641735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641699    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f22d18d9018dbc9c1a4c9494ab8fa726-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal\" (UID: \"f22d18d9018dbc9c1a4c9494ab8fa726\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.641859 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.641742    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4c4d2eab85087dd7f7d7ed97e5b0c0e8-config\") pod \"kube-apiserver-proxy-ip-10-0-137-160.ec2.internal\" (UID: \"4c4d2eab85087dd7f7d7ed97e5b0c0e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.727080 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.727013    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.817580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.817539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.821637 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:49.821455    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:49.827987 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.827955    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:49.928517 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:49.928482    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.029061 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:50.028987    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.129501 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:50.129468    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.183937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.183907    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 03:19:50.184432 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.184047    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 03:19:50.194756 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.194733    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:50.230292 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:50.230266    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.248956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.248919    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 03:14:50 +0000 UTC" deadline="2028-03-10 09:45:28.480452591 +0000 UTC"
Jul 08 03:19:50.248956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.248953    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14670h25m38.231505988s"
Jul 08 03:19:50.331494 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:50.331471    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.431831 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:50.431797    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-160.ec2.internal\" not found"
Jul 08 03:19:50.468139 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:50.468110    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4d2eab85087dd7f7d7ed97e5b0c0e8.slice/crio-a15f4a798e7fe5fa5ba7c128ed409307450284fa8480aa528877d37b8a947a3b WatchSource:0}: Error finding container a15f4a798e7fe5fa5ba7c128ed409307450284fa8480aa528877d37b8a947a3b: Status 404 returned error can't find the container with id a15f4a798e7fe5fa5ba7c128ed409307450284fa8480aa528877d37b8a947a3b
Jul 08 03:19:50.468450 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:50.468433    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf22d18d9018dbc9c1a4c9494ab8fa726.slice/crio-80a927b37cc36e8cdd1da9c72e302ea716e86e45c6e995687c386db0affcf22d WatchSource:0}: Error finding container 80a927b37cc36e8cdd1da9c72e302ea716e86e45c6e995687c386db0affcf22d: Status 404 returned error can't find the container with id 80a927b37cc36e8cdd1da9c72e302ea716e86e45c6e995687c386db0affcf22d
Jul 08 03:19:50.474657 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.474629    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:19:50.493513 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.493495    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:50.538700 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.538681    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:50.548945 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.548928    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 03:19:50.550841 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.550829    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal"
Jul 08 03:19:50.560839 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.560824    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 03:19:50.724068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:50.723982    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:51.098152 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.097927    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 03:19:51.218020 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.217987    2563 apiserver.go:52] "Watching apiserver"
Jul 08 03:19:51.223289 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.223258    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 03:19:51.225465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.225438    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-skcm9","kube-system/global-pull-secret-syncer-h6zkr","openshift-dns/node-resolver-ffj7r","openshift-image-registry/node-ca-hpqlc","openshift-multus/multus-additional-cni-plugins-wf9lr","openshift-network-operator/iptables-alerter-jvdrr","openshift-ovn-kubernetes/ovnkube-node-h5lx7","kube-system/konnectivity-agent-jgnfd","kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725","openshift-cluster-node-tuning-operator/tuned-g7576","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal","openshift-multus/multus-47nlr","openshift-multus/network-metrics-daemon-v5fdt"]
Jul 08 03:19:51.228688 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.228662    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.232964 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.232938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.233479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.233459    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 03:19:51.233602 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.233505    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 03:19:51.233662 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.233644    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.234411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.234391    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 03:19:51.234501 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.234429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 03:19:51.234501 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.234449    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.234682 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.234663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-nls8h\""
Jul 08 03:19:51.235054 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.235035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.236057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.235813    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kjj22\""
Jul 08 03:19:51.236182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.236071    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 03:19:51.236314 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.236294    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 03:19:51.236436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.236420    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.238972 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.238948    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 03:19:51.241485 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.241466    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.243589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.243315    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.243589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.243412    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.243589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.243499    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9nbsk\""
Jul 08 03:19:51.243976 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.243954    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.245973 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.245943    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.246169 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.246152    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.246405 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.246383    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zhvw9\""
Jul 08 03:19:51.246635 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.246621    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 03:19:51.247004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.246981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.249826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.249423    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 03:19:51.249826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.249814    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 03:19:51.250085 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.250062    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kjxsp\""
Jul 08 03:19:51.255088 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255059    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 03:19:51.255182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/407827ad-6c24-4b50-80b2-a488d63f499f-ovn-node-metrics-cert\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255139    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-bin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255302 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255189    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-hostroot\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255302 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-socket-dir-parent\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255302 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-multus-certs\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255445 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9478073e-2c45-4a5f-838f-ae62733ea6ee-hosts-file\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.255445 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-systemd-units\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255445 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-slash\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255445 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-netns\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-etc-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-log-socket\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255583    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-netd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.255633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-etc-kubernetes\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-var-lib-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255693    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-node-log\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255725    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255759    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-k8s-cni-cncf-io\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-netns\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255817    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9478073e-2c45-4a5f-838f-ae62733ea6ee-tmp-dir\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-systemd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.255918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkkx5\" (UniqueName: \"kubernetes.io/projected/407827ad-6c24-4b50-80b2-a488d63f499f-kube-api-access-jkkx5\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr49h\" (UniqueName: \"kubernetes.io/projected/77286f6d-68a7-41d3-98d2-dd0201a623ea-kube-api-access-dr49h\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.255989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-kubelet\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-ovn\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-config\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-os-release\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-multus\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256161    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-kubelet\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-env-overrides\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-script-lib\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256282    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-cnibin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256316    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-daemon-config\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9slft\" (UniqueName: \"kubernetes.io/projected/9478073e-2c45-4a5f-838f-ae62733ea6ee-kube-api-access-9slft\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-bin\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-system-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256510    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-cni-binary-copy\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.256986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.256547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-conf-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.258120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.257615    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rnds2\""
Jul 08 03:19:51.258120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.257659    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.258120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.257935    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 03:19:51.258120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.258037    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.260383 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.260347    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.260491 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.260445    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:19:51.266479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.266461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:51.266567 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.266528    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:19:51.268746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.268601    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.268746 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.268707    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:19:51.271810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.271608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.274252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.274233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.274442 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.274417    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.274517 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.274457    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 03:19:51.274517 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.274500    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-g2rp7\""
Jul 08 03:19:51.274623 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.274433    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.276485 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.276427    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:19:51.276798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.276782    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 03:19:51.276905 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.276884    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5j9m7\""
Jul 08 03:19:51.277506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.277463    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4b45n"
Jul 08 03:19:51.279231 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.277865    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.279965 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.279944    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 03:19:51.280433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.280413    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 03:19:51.280593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.280579    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8kctr\""
Jul 08 03:19:51.287146 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.287125    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4b45n"
Jul 08 03:19:51.339961 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.339935    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 03:19:51.356849 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-slash\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.356849 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356807    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-netns\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.356849 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-netd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356891    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-netns\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-etc-kubernetes\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356891    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-slash\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356960    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-netd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356976    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-etc-kubernetes\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.356990    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-cnibin\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357042    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a517e8cd-aa24-49db-96da-2a7d061a806e-host-slash\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-modprobe-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357082    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-var-lib-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357112    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357173    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-var-lib-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357184    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-k8s-cni-cncf-io\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357237    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a517e8cd-aa24-49db-96da-2a7d061a806e-iptables-alerter-script\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-registration-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357288    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysconfig\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357312    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357311    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-k8s-cni-cncf-io\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357338    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-var-lib-kubelet\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-systemd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr49h\" (UniqueName: \"kubernetes.io/projected/77286f6d-68a7-41d3-98d2-dd0201a623ea-kube-api-access-dr49h\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-systemd\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357418    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-systemd\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357436    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-sys\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-kubelet\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.357612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-config\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-os-release\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357604    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-kubelet\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357628    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-os-release\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357654    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-tmp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxglp\" (UniqueName: \"kubernetes.io/projected/93138aa4-acc2-4506-ac94-661c377ed192-kube-api-access-rxglp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357747    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-kubelet-config\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/969cb854-5ab3-4f0a-908d-762dc8b0955a-agent-certs\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-env-overrides\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-daemon-config\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.357958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-lib-modules\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/969cb854-5ab3-4f0a-908d-762dc8b0955a-konnectivity-ca\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-bin\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358219    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-system-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.358483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-conf-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358301    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358308    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-config\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358324    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-env-overrides\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-conf\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358378    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-system-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358397    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-dbus\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358440    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-conf-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358451    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358474    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-cni-bin\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/407827ad-6c24-4b50-80b2-a488d63f499f-ovn-node-metrics-cert\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-os-release\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-sys-fs\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxqft\" (UniqueName: \"kubernetes.io/projected/848ea393-51f5-40f4-919f-7b147f20ff22-kube-api-access-kxqft\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-socket-dir-parent\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-multus-certs\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-socket-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.359303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj4dj\" (UniqueName: \"kubernetes.io/projected/850ebebc-f603-49c4-9fba-08daf60e0b7d-kube-api-access-fj4dj\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358768    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-socket-dir-parent\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33643870-0989-4f93-9e49-61cd14adfd7f-host\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358797    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-multus-certs\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-systemd-units\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-etc-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358854    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-log-socket\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358863    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-systemd-units\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358871    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358901    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-etc-openvswitch\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358917    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-log-socket\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33643870-0989-4f93-9e49-61cd14adfd7f-serviceca\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358944    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-host-run-ovn-kubernetes\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-node-log\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358985    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-node-log\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358991    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-daemon-config\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359006    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-netns\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-run-netns\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.360183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.358993    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9478073e-2c45-4a5f-838f-ae62733ea6ee-tmp-dir\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359055    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2vxv\" (UniqueName: \"kubernetes.io/projected/a517e8cd-aa24-49db-96da-2a7d061a806e-kube-api-access-k2vxv\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359077    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9nqw\" (UniqueName: \"kubernetes.io/projected/33643870-0989-4f93-9e49-61cd14adfd7f-kube-api-access-j9nqw\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359102    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkkx5\" (UniqueName: \"kubernetes.io/projected/407827ad-6c24-4b50-80b2-a488d63f499f-kube-api-access-jkkx5\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359162    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-device-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-ovn\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-multus\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-kubelet\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359240    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-multus-cni-dir\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359287    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/407827ad-6c24-4b50-80b2-a488d63f499f-run-ovn\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359288    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-kubelet\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-kubernetes\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359306    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-multus\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-script-lib\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359342    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9478073e-2c45-4a5f-838f-ae62733ea6ee-tmp-dir\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.361124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-cnibin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9slft\" (UniqueName: \"kubernetes.io/projected/9478073e-2c45-4a5f-838f-ae62733ea6ee-kube-api-access-9slft\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359413    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q9rw\" (UniqueName: \"kubernetes.io/projected/ba1ffb6a-2588-4411-b936-433be263812f-kube-api-access-9q9rw\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359422    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-cnibin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-cni-binary-copy\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-system-cni-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359518    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-run\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359571    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-etc-tuned\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359624    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-bin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-hostroot\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359678    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-host-var-lib-cni-bin\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359687    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359733    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/77286f6d-68a7-41d3-98d2-dd0201a623ea-hostroot\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-host\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.362050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359764    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.362896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9478073e-2c45-4a5f-838f-ae62733ea6ee-hosts-file\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.362896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359870    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77286f6d-68a7-41d3-98d2-dd0201a623ea-cni-binary-copy\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.362896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359894    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9478073e-2c45-4a5f-838f-ae62733ea6ee-hosts-file\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.362896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.359901    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/407827ad-6c24-4b50-80b2-a488d63f499f-ovnkube-script-lib\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.364964 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.364937    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/407827ad-6c24-4b50-80b2-a488d63f499f-ovn-node-metrics-cert\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.366471 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.366411    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr49h\" (UniqueName: \"kubernetes.io/projected/77286f6d-68a7-41d3-98d2-dd0201a623ea-kube-api-access-dr49h\") pod \"multus-47nlr\" (UID: \"77286f6d-68a7-41d3-98d2-dd0201a623ea\") " pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.368355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.368334    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkkx5\" (UniqueName: \"kubernetes.io/projected/407827ad-6c24-4b50-80b2-a488d63f499f-kube-api-access-jkkx5\") pod \"ovnkube-node-h5lx7\" (UID: \"407827ad-6c24-4b50-80b2-a488d63f499f\") " pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.368449 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.368351    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9slft\" (UniqueName: \"kubernetes.io/projected/9478073e-2c45-4a5f-838f-ae62733ea6ee-kube-api-access-9slft\") pod \"node-resolver-ffj7r\" (UID: \"9478073e-2c45-4a5f-838f-ae62733ea6ee\") " pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.384722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.384580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal" event={"ID":"f22d18d9018dbc9c1a4c9494ab8fa726","Type":"ContainerStarted","Data":"80a927b37cc36e8cdd1da9c72e302ea716e86e45c6e995687c386db0affcf22d"}
Jul 08 03:19:51.386220 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.386184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal" event={"ID":"4c4d2eab85087dd7f7d7ed97e5b0c0e8","Type":"ContainerStarted","Data":"a15f4a798e7fe5fa5ba7c128ed409307450284fa8480aa528877d37b8a947a3b"}
Jul 08 03:19:51.460695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460653    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-system-cni-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460773    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-run\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460781    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-system-cni-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-etc-tuned\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.460884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460870    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-host\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460921    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-etc-selinux\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.460980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-cnibin\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a517e8cd-aa24-49db-96da-2a7d061a806e-host-slash\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.461062    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-modprobe-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a517e8cd-aa24-49db-96da-2a7d061a806e-iptables-alerter-script\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.461123    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:51.961105511 +0000 UTC m=+3.069451433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-registration-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysconfig\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461179    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461220    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-var-lib-kubelet\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461243    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-systemd\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-sys\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.461274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461278    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-tmp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxglp\" (UniqueName: \"kubernetes.io/projected/93138aa4-acc2-4506-ac94-661c377ed192-kube-api-access-rxglp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-kubelet-config\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/969cb854-5ab3-4f0a-908d-762dc8b0955a-agent-certs\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461378    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-lib-modules\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461411    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/969cb854-5ab3-4f0a-908d-762dc8b0955a-konnectivity-ca\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-conf\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-dbus\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-os-release\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461520    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-sys-fs\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxqft\" (UniqueName: \"kubernetes.io/projected/848ea393-51f5-40f4-919f-7b147f20ff22-kube-api-access-kxqft\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-socket-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461606    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj4dj\" (UniqueName: \"kubernetes.io/projected/850ebebc-f603-49c4-9fba-08daf60e0b7d-kube-api-access-fj4dj\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33643870-0989-4f93-9e49-61cd14adfd7f-host\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.462118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461664    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33643870-0989-4f93-9e49-61cd14adfd7f-serviceca\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2vxv\" (UniqueName: \"kubernetes.io/projected/a517e8cd-aa24-49db-96da-2a7d061a806e-kube-api-access-k2vxv\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461703    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a517e8cd-aa24-49db-96da-2a7d061a806e-iptables-alerter-script\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461721    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9nqw\" (UniqueName: \"kubernetes.io/projected/33643870-0989-4f93-9e49-61cd14adfd7f-kube-api-access-j9nqw\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-device-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-kubernetes\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q9rw\" (UniqueName: \"kubernetes.io/projected/ba1ffb6a-2588-4411-b936-433be263812f-kube-api-access-9q9rw\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461813    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-cnibin\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461952    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461995    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a517e8cd-aa24-49db-96da-2a7d061a806e-host-slash\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.462104    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-run\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.462107    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-modprobe-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.462667    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.462698    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/969cb854-5ab3-4f0a-908d-762dc8b0955a-konnectivity-ca\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.462981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.462798    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-lib-modules\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463127    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463124    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-var-lib-kubelet\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463193    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-registration-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463264    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysconfig\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463267    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-conf\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463354    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-dbus\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.463392    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ba1ffb6a-2588-4411-b936-433be263812f-os-release\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-sysctl-d\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.463447    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:19:51.963429853 +0000 UTC m=+3.071775812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463449    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-sys-fs\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463509    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-host\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.463741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463740    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-socket-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463798    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-device-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463850    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-kubernetes\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.461778    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/850ebebc-f603-49c4-9fba-08daf60e0b7d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.463914    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/496ab302-a93d-4c2b-96ba-7632f4c92c9c-kubelet-config\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.464099    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33643870-0989-4f93-9e49-61cd14adfd7f-host\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.464425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.464372    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-sys\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.464716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.464432    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/93138aa4-acc2-4506-ac94-661c377ed192-etc-systemd\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.464716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.464545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/33643870-0989-4f93-9e49-61cd14adfd7f-serviceca\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.465123 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.465008    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ba1ffb6a-2588-4411-b936-433be263812f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.466607 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.466575    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-tmp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.466994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.466971    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/93138aa4-acc2-4506-ac94-661c377ed192-etc-tuned\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.467311 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.467294    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/969cb854-5ab3-4f0a-908d-762dc8b0955a-agent-certs\") pod \"konnectivity-agent-jgnfd\" (UID: \"969cb854-5ab3-4f0a-908d-762dc8b0955a\") " pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.470953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.470930    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q9rw\" (UniqueName: \"kubernetes.io/projected/ba1ffb6a-2588-4411-b936-433be263812f-kube-api-access-9q9rw\") pod \"multus-additional-cni-plugins-wf9lr\" (UID: \"ba1ffb6a-2588-4411-b936-433be263812f\") " pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.473326 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.473305    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:51.473326 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.473330    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:51.473483 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.473345    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:51.473483 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.473393    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:19:51.97338094 +0000 UTC m=+3.081726860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:51.477426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.477405    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxqft\" (UniqueName: \"kubernetes.io/projected/848ea393-51f5-40f4-919f-7b147f20ff22-kube-api-access-kxqft\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.477972 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.477949    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9nqw\" (UniqueName: \"kubernetes.io/projected/33643870-0989-4f93-9e49-61cd14adfd7f-kube-api-access-j9nqw\") pod \"node-ca-hpqlc\" (UID: \"33643870-0989-4f93-9e49-61cd14adfd7f\") " pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.478055 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.477975    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj4dj\" (UniqueName: \"kubernetes.io/projected/850ebebc-f603-49c4-9fba-08daf60e0b7d-kube-api-access-fj4dj\") pod \"aws-ebs-csi-driver-node-6q725\" (UID: \"850ebebc-f603-49c4-9fba-08daf60e0b7d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.478104 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.478084    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxglp\" (UniqueName: \"kubernetes.io/projected/93138aa4-acc2-4506-ac94-661c377ed192-kube-api-access-rxglp\") pod \"tuned-g7576\" (UID: \"93138aa4-acc2-4506-ac94-661c377ed192\") " pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.479648 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.479615    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2vxv\" (UniqueName: \"kubernetes.io/projected/a517e8cd-aa24-49db-96da-2a7d061a806e-kube-api-access-k2vxv\") pod \"iptables-alerter-jvdrr\" (UID: \"a517e8cd-aa24-49db-96da-2a7d061a806e\") " pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.544913 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.544879    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:19:51.555645 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.555612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-47nlr"
Jul 08 03:19:51.568699 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.568674    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ffj7r"
Jul 08 03:19:51.576716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.576694    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hpqlc"
Jul 08 03:19:51.586525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.586499    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf9lr"
Jul 08 03:19:51.594399 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.594378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-jvdrr"
Jul 08 03:19:51.602307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.602283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725"
Jul 08 03:19:51.617343 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.617268    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-g7576"
Jul 08 03:19:51.622157 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.622117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:19:51.966024 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.965928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:51.966024 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:51.965973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:51.966273 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.966110    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:51.966273 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.966113    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:51.966273 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.966200    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:52.96616244 +0000 UTC m=+4.074508360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:51.966273 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:51.966221    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:19:52.96621145 +0000 UTC m=+4.074557385 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:52.066577 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.066537    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:52.066770 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.066693    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:52.066770 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.066718    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:52.066770 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.066734    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:52.066925 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.066801    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:19:53.066783332 +0000 UTC m=+4.175129251 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:52.290263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.290183    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 03:14:52 +0000 UTC" deadline="2028-02-07 20:21:26.680548527 +0000 UTC"
Jul 08 03:19:52.290263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.290214    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13913h1m34.390337543s"
Jul 08 03:19:52.379213 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.379182    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:52.379401 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.379195    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:52.379401 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.379293    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:19:52.379528 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.379398    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:19:52.594767 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:52.594736    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba1ffb6a_2588_4411_b936_433be263812f.slice/crio-9b66c759aca93a2389e53262ab5ef92c2547088034addfea550c824158e80ff2 WatchSource:0}: Error finding container 9b66c759aca93a2389e53262ab5ef92c2547088034addfea550c824158e80ff2: Status 404 returned error can't find the container with id 9b66c759aca93a2389e53262ab5ef92c2547088034addfea550c824158e80ff2
Jul 08 03:19:52.598133 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:52.597180    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93138aa4_acc2_4506_ac94_661c377ed192.slice/crio-e2367e1d69b82b73711882a2238d5332d43216f84171d90ad660feb3d47df8f8 WatchSource:0}: Error finding container e2367e1d69b82b73711882a2238d5332d43216f84171d90ad660feb3d47df8f8: Status 404 returned error can't find the container with id e2367e1d69b82b73711882a2238d5332d43216f84171d90ad660feb3d47df8f8
Jul 08 03:19:52.600940 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:52.600920    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9478073e_2c45_4a5f_838f_ae62733ea6ee.slice/crio-adab0a11ef79a114b36e4a865d3e624e42a37bc66dd56a39d163fa0d3c695dd1 WatchSource:0}: Error finding container adab0a11ef79a114b36e4a865d3e624e42a37bc66dd56a39d163fa0d3c695dd1: Status 404 returned error can't find the container with id adab0a11ef79a114b36e4a865d3e624e42a37bc66dd56a39d163fa0d3c695dd1
Jul 08 03:19:52.605605 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:52.605503    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77286f6d_68a7_41d3_98d2_dd0201a623ea.slice/crio-423cd3bd0fed4fc07553dba11fbf4a11c310fea0db03e1abd4c59292314a3983 WatchSource:0}: Error finding container 423cd3bd0fed4fc07553dba11fbf4a11c310fea0db03e1abd4c59292314a3983: Status 404 returned error can't find the container with id 423cd3bd0fed4fc07553dba11fbf4a11c310fea0db03e1abd4c59292314a3983
Jul 08 03:19:52.620992 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:19:52.620963    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969cb854_5ab3_4f0a_908d_762dc8b0955a.slice/crio-2eb65512c702dbdd6a21fef05d6489987acb99519d8b430c1cd4034a49fec99c WatchSource:0}: Error finding container 2eb65512c702dbdd6a21fef05d6489987acb99519d8b430c1cd4034a49fec99c: Status 404 returned error can't find the container with id 2eb65512c702dbdd6a21fef05d6489987acb99519d8b430c1cd4034a49fec99c
Jul 08 03:19:52.974252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.974036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:52.974252 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.974230    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:52.974452 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:52.974267    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:52.974452 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.974299    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:19:54.974280995 +0000 UTC m=+6.082626931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:52.974452 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.974340    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:52.974452 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:52.974381    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:54.974368987 +0000 UTC m=+6.082714906 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:53.074999 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.074907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:53.075141 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:53.075099    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:53.075141 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:53.075123    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:53.075141 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:53.075135    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:53.075332 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:53.075218    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:19:55.075178825 +0000 UTC m=+6.183524760 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:53.290831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.290731    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 03:14:52 +0000 UTC" deadline="2027-12-16 14:07:13.771907516 +0000 UTC"
Jul 08 03:19:53.290831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.290787    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12634h47m20.481123946s"
Jul 08 03:19:53.382210 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.381667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:53.382210 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:53.381792    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:19:53.409408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.409367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"76e82d470cc17b4567207015a400967483ae419946f6dad2883f7824da9dbbe8"}
Jul 08 03:19:53.427638 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.427552    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jvdrr" event={"ID":"a517e8cd-aa24-49db-96da-2a7d061a806e","Type":"ContainerStarted","Data":"c1c5f9e3907c7eef4b864ae0a62e308a661454280523f581af64ee63c2be6624"}
Jul 08 03:19:53.435040 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.434925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ffj7r" event={"ID":"9478073e-2c45-4a5f-838f-ae62733ea6ee","Type":"ContainerStarted","Data":"adab0a11ef79a114b36e4a865d3e624e42a37bc66dd56a39d163fa0d3c695dd1"}
Jul 08 03:19:53.451955 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.451170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal" event={"ID":"4c4d2eab85087dd7f7d7ed97e5b0c0e8","Type":"ContainerStarted","Data":"8b824b1d2397fcc7bfab93e91432f2e71f2d5d62623f4300935d83145c6aae24"}
Jul 08 03:19:53.461348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.461283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpqlc" event={"ID":"33643870-0989-4f93-9e49-61cd14adfd7f","Type":"ContainerStarted","Data":"3cf003caff567076b72c10739d2472d6bd56af8ae4d89644b91d9f06b2c8953c"}
Jul 08 03:19:53.467812 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.467110    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-160.ec2.internal" podStartSLOduration=2.4670904289999998 podStartE2EDuration="2.467090429s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:19:53.466413234 +0000 UTC m=+4.574759216" watchObservedRunningTime="2026-07-08 03:19:53.467090429 +0000 UTC m=+4.575436373"
Jul 08 03:19:53.480198 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.480144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725" event={"ID":"850ebebc-f603-49c4-9fba-08daf60e0b7d","Type":"ContainerStarted","Data":"c74ec7aead63ebddf1c7300d21f2e9efacfeb1db070978d97c0c270669ffee40"}
Jul 08 03:19:53.487120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.487089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47nlr" event={"ID":"77286f6d-68a7-41d3-98d2-dd0201a623ea","Type":"ContainerStarted","Data":"423cd3bd0fed4fc07553dba11fbf4a11c310fea0db03e1abd4c59292314a3983"}
Jul 08 03:19:53.494944 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.494911    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g7576" event={"ID":"93138aa4-acc2-4506-ac94-661c377ed192","Type":"ContainerStarted","Data":"e2367e1d69b82b73711882a2238d5332d43216f84171d90ad660feb3d47df8f8"}
Jul 08 03:19:53.500127 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.500097    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerStarted","Data":"9b66c759aca93a2389e53262ab5ef92c2547088034addfea550c824158e80ff2"}
Jul 08 03:19:53.518522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.518491    2563 generic.go:356] "Generic (PLEG): container finished" podID="f22d18d9018dbc9c1a4c9494ab8fa726" containerID="cac73221b8d082501e1c50b6a0388f5a3ffdc7c34ddc6eb2473cc1b3aaa8245b" exitCode=0
Jul 08 03:19:53.518634 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.518604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal" event={"ID":"f22d18d9018dbc9c1a4c9494ab8fa726","Type":"ContainerDied","Data":"cac73221b8d082501e1c50b6a0388f5a3ffdc7c34ddc6eb2473cc1b3aaa8245b"}
Jul 08 03:19:53.523273 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:53.523242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgnfd" event={"ID":"969cb854-5ab3-4f0a-908d-762dc8b0955a","Type":"ContainerStarted","Data":"2eb65512c702dbdd6a21fef05d6489987acb99519d8b430c1cd4034a49fec99c"}
Jul 08 03:19:54.382678 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:54.382638    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:54.383059 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.382795    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:19:54.383059 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:54.382964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:54.383131 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.383058    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:19:54.538324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:54.538279    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal" event={"ID":"f22d18d9018dbc9c1a4c9494ab8fa726","Type":"ContainerStarted","Data":"e7d1efd229ad541b91798c30ac2fb216bf1cabcfed867fd26f699c96edcdd0a9"}
Jul 08 03:19:54.996583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:54.996544    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:54.996794 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:54.996599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:54.996794 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.996721    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:54.996794 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.996779    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:19:58.99675854 +0000 UTC m=+10.105104484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:54.997163 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.997138    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:54.997341 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:54.997187    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:19:58.997174239 +0000 UTC m=+10.105520173 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:55.097550 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:55.097485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:55.097756 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:55.097692    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:55.097756 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:55.097714    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:55.097756 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:55.097729    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:55.097923 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:55.097788    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:19:59.097770356 +0000 UTC m=+10.206116291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:55.381971 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:55.381937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:55.382189 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:55.382065    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:19:56.379534 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:56.378967    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:56.379534 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:56.379063    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:56.379534 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:56.379195    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:19:56.379534 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:56.379346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:19:57.379164 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:57.379080    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:57.379363 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:57.379241    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:19:58.379254 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:58.379215    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:58.379254 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:58.379230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:58.379748 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:58.379344    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:19:58.379748 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:58.379531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:19:59.028677 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:59.028635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:19:59.028876 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:59.028705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:19:59.028876 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.028832    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:59.029021 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.028874    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:59.029021 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.028919    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:20:07.028897569 +0000 UTC m=+18.137243493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:19:59.029132 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.029069    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:07.029026708 +0000 UTC m=+18.137372639 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:19:59.129649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:59.129614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:59.129840 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.129749    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:19:59.129840 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.129766    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:19:59.129840 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.129775    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:59.129840 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.129838    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:07.129817294 +0000 UTC m=+18.238163214 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:19:59.381326 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:19:59.381295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:19:59.381791 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:19:59.381390    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:00.379441 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:00.379408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:00.379625 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:00.379417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:00.379625 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:00.379546    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:00.379625 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:00.379593    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:01.379381 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:01.379354    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:01.379828 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:01.379478    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:02.378620 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:02.378579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:02.378802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:02.378597    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:02.378802 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:02.378725    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:02.378917 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:02.378823    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:03.379810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.379251    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:03.379810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:03.379373    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:03.570483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.569628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-g7576" event={"ID":"93138aa4-acc2-4506-ac94-661c377ed192","Type":"ContainerStarted","Data":"b8e32cdbb90fd4d4901792979424976732d95ab0308326b326082990956d7bf2"}
Jul 08 03:20:03.572508 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.572451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgnfd" event={"ID":"969cb854-5ab3-4f0a-908d-762dc8b0955a","Type":"ContainerStarted","Data":"7bf57558c8112c54ae5a804b6cd7de7481b548a0f00f51f47d74f4bff3b593db"}
Jul 08 03:20:03.574808 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.574379    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ffj7r" event={"ID":"9478073e-2c45-4a5f-838f-ae62733ea6ee","Type":"ContainerStarted","Data":"2773c5c0f1a551690a1a34b4c738ed9ed997f4a5d28b4bcb6f223e42926288c5"}
Jul 08 03:20:03.588392 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.588345    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-g7576" podStartSLOduration=2.804150495 podStartE2EDuration="13.588300471s" totalImagesPullingTime="10.784149976s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:03.588192876 +0000 UTC m=+14.696538816" watchObservedRunningTime="2026-07-08 03:20:03.588300471 +0000 UTC m=+14.696646410"
Jul 08 03:20:03.588579 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.588545    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-160.ec2.internal" podStartSLOduration=12.588530726 podStartE2EDuration="12.588530726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:19:54.554023053 +0000 UTC m=+5.662368997" watchObservedRunningTime="2026-07-08 03:20:03.588530726 +0000 UTC m=+14.696876667"
Jul 08 03:20:03.622283 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.622233    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jgnfd" podStartSLOduration=2.931376764 podStartE2EDuration="13.622194404s" totalImagesPullingTime="10.69081764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:03.607358379 +0000 UTC m=+14.715704318" watchObservedRunningTime="2026-07-08 03:20:03.622194404 +0000 UTC m=+14.730540345"
Jul 08 03:20:03.995187 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.995101    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:20:03.995732 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:03.995715    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:20:04.010273 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.010224    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ffj7r" podStartSLOduration=3.275737694 podStartE2EDuration="14.010205949s" totalImagesPullingTime="10.734468255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:03.623480887 +0000 UTC m=+14.731826840" watchObservedRunningTime="2026-07-08 03:20:04.010205949 +0000 UTC m=+15.118551890"
Jul 08 03:20:04.379135 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.379101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:04.379135 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.379162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:04.379352 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:04.379244    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:04.379405 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:04.379364    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:04.581266 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.581232    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="64867234746a8569744b45ddaef292cd9fcb1961a0761d7b4dccff172ad2ae1e" exitCode=0
Jul 08 03:20:04.581875 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.581334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"64867234746a8569744b45ddaef292cd9fcb1961a0761d7b4dccff172ad2ae1e"}
Jul 08 03:20:04.584289 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.583859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hpqlc" event={"ID":"33643870-0989-4f93-9e49-61cd14adfd7f","Type":"ContainerStarted","Data":"2d2711595bd62f2be96a166b75ef1eb22d001353f25bd8201444600cc718fbb1"}
Jul 08 03:20:04.587113 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:04.586779    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725" event={"ID":"850ebebc-f603-49c4-9fba-08daf60e0b7d","Type":"ContainerStarted","Data":"d8ee3b84ec8b9a6cd6759e9f1214112d4586c8247ebc73c4f40f8f7ee3f12546"}
Jul 08 03:20:05.381340 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:05.381305    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:05.381506 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:05.381433    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:05.590337 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:05.590303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-jvdrr" event={"ID":"a517e8cd-aa24-49db-96da-2a7d061a806e","Type":"ContainerStarted","Data":"b899ab60e3a70a9b0ef987d05e45c223abf4e95ddc30a6e420fedfbd8370d595"}
Jul 08 03:20:05.590884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:05.590477    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 08 03:20:05.606292 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:05.606242    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hpqlc" podStartSLOduration=4.880444962 podStartE2EDuration="15.606223227s" totalImagesPullingTime="10.725778265s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:04.620939576 +0000 UTC m=+15.729285517" watchObservedRunningTime="2026-07-08 03:20:05.606223227 +0000 UTC m=+16.714569169"
Jul 08 03:20:06.379009 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:06.378977    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:06.379202 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:06.379114    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:06.379202 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:06.379180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:06.379344 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:06.379306    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:07.095852 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:07.095818    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:07.096362 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:07.095879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:07.096362 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.095975    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:07.096362 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.096008    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:07.096362 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.096055    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:20:23.096034707 +0000 UTC m=+34.204380643 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:07.096362 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.096076    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:23.096064687 +0000 UTC m=+34.204410604 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:07.196638 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:07.196607    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:07.196811 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.196778    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:20:07.196811 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.196797    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:20:07.196811 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.196810    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:07.196975 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.196868    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:23.19684867 +0000 UTC m=+34.305194587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:07.378917 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:07.378838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:07.379067 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:07.378957    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:08.379248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:08.379214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:08.379721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:08.379214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:08.379721 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:08.379329    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:08.379721 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:08.379417    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:09.380481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:09.380238    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:09.380894 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:09.380580    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:09.779080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:09.778959    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:20:09.779245 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:09.779097    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 08 03:20:09.779899 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:09.779877    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jgnfd"
Jul 08 03:20:09.795913 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:09.795863    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-jvdrr" podStartSLOduration=9.07086689 podStartE2EDuration="19.795849033s" totalImagesPullingTime="10.724982143s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:05.605455389 +0000 UTC m=+16.713801329" watchObservedRunningTime="2026-07-08 03:20:09.795849033 +0000 UTC m=+20.904195004"
Jul 08 03:20:10.379258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:10.379224    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:10.379422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:10.379231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:10.379422 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:10.379352    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:10.379538 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:10.379457    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:11.379261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:11.379231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:11.379917 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:11.379353    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:12.379270 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:12.379231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:12.379769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:12.379248    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:12.379769 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:12.379349    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:12.379769 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:12.379450    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:13.379363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.379333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:13.379773 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:13.379457    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:13.420413 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.420387    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 03:20:13.608738 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.608708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725" event={"ID":"850ebebc-f603-49c4-9fba-08daf60e0b7d","Type":"ContainerStarted","Data":"aa8dc683e920206a9c5eeb6d5f22f8111bc0396609a5f9328c38df81d70f3b5c"}
Jul 08 03:20:13.610868 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.610445    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-47nlr" event={"ID":"77286f6d-68a7-41d3-98d2-dd0201a623ea","Type":"ContainerStarted","Data":"399f09c8481177936c1e6d277f2b154098ac6b2aed8e0720b2cd7497c017d514"}
Jul 08 03:20:13.612737 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.612434    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:20:13.613284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.613051    2563 generic.go:356] "Generic (PLEG): container finished" podID="407827ad-6c24-4b50-80b2-a488d63f499f" containerID="923491bbad763cab2e1d32fd8d6e66c44f0631844db916cc3fc2172407194ba8" exitCode=1
Jul 08 03:20:13.613284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.613094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerDied","Data":"923491bbad763cab2e1d32fd8d6e66c44f0631844db916cc3fc2172407194ba8"}
Jul 08 03:20:13.613284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.613202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"0a7e826dccd003406f622a4f458456d1cecf38e899a8b5bb4c6549695dca65e5"}
Jul 08 03:20:13.630410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:13.630198    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-47nlr" podStartSLOduration=2.93310049 podStartE2EDuration="23.630178519s" totalImagesPullingTime="20.697078029s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:13.629563303 +0000 UTC m=+24.737909243" watchObservedRunningTime="2026-07-08 03:20:13.630178519 +0000 UTC m=+24.738524463"
Jul 08 03:20:14.318277 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.318168    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T03:20:13.420407979Z","UUID":"8a6a8898-961d-41c0-b0d9-7cff46490871","Handler":null,"Name":"","Endpoint":""}
Jul 08 03:20:14.320626 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.320604    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 03:20:14.320757 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.320649    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 03:20:14.378784 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.378746    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:14.378885 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.378803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:14.378885 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:14.378867    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:14.379046 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:14.379016    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:14.618858 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.618788    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:20:14.619459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.619217    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"f2111739377e3284be0a7c866f5d3d5fb0f4441703e096fba90645da04a326ed"}
Jul 08 03:20:14.619459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.619299    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"c289ded90d07e48162c33fb5321cee8208d83f83fcd82e62b99de8411bf10bc6"}
Jul 08 03:20:14.619459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.619348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"53cfd1747e69efcb3fae707482271eb5791ff98bcc91ffc4dc0158bce68d519b"}
Jul 08 03:20:14.619459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.619381    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"fe57c5433d76c9a9b5050abe4c89c4c3b3985df71576a96e22acd0f81f2e8535"}
Jul 08 03:20:14.621698 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.621667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725" event={"ID":"850ebebc-f603-49c4-9fba-08daf60e0b7d","Type":"ContainerStarted","Data":"ca3f37f91e835c3bb5e084aa91a4a9c0ee6ef6fea44926dc57a834833e82a5f7"}
Jul 08 03:20:14.640938 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:14.640892    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6q725" podStartSLOduration=3.316809074 podStartE2EDuration="24.640878566s" totalImagesPullingTime="21.324069492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:14.640639773 +0000 UTC m=+25.748985713" watchObservedRunningTime="2026-07-08 03:20:14.640878566 +0000 UTC m=+25.749224506"
Jul 08 03:20:15.379481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:15.379445    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:15.379663 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:15.379558    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:16.379358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:16.379325    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:16.380013 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:16.379446    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:16.380013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:16.379328    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:16.380013 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:16.379842    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:16.629405 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:16.629333    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:20:16.629788 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:16.629763    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"c2bab708514191387c039c9c8912561253c3a0705ee3a72e126711a1a1aff04c"}
Jul 08 03:20:17.379260 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:17.379229    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:17.379435 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:17.379341    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:17.634093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:17.634004    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerStarted","Data":"72250868f16604ff147a499de8d4320a38628c741ffdefd1a7d8a42e0d608c3e"}
Jul 08 03:20:18.379076 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.378905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:18.379266 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.378978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:18.379266 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:18.379176    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:18.379266 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:18.379244    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:18.638226 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.638126    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="72250868f16604ff147a499de8d4320a38628c741ffdefd1a7d8a42e0d608c3e" exitCode=0
Jul 08 03:20:18.638226 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.638172    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"72250868f16604ff147a499de8d4320a38628c741ffdefd1a7d8a42e0d608c3e"}
Jul 08 03:20:18.641963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.641947    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:20:18.642362 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.642336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"6a9d7b976347a8869ac19eace9c91f1a5ab638d443ff4847a44e6d5d406bf708"}
Jul 08 03:20:18.642597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.642572    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:18.642706 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.642648    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:18.642908 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.642826    2563 scope.go:117] "RemoveContainer" containerID="923491bbad763cab2e1d32fd8d6e66c44f0631844db916cc3fc2172407194ba8"
Jul 08 03:20:18.660957 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:18.660910    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:19.379754 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.379724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:19.380024 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:19.379839    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:19.646581 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.646496    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="732c86d755f18f8ec733576b9ba41b7586d62fb41c5cbcdfe5a4710d2244a337" exitCode=0
Jul 08 03:20:19.647000 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.646571    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"732c86d755f18f8ec733576b9ba41b7586d62fb41c5cbcdfe5a4710d2244a337"}
Jul 08 03:20:19.650671 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.650651    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:20:19.651120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.651050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" event={"ID":"407827ad-6c24-4b50-80b2-a488d63f499f","Type":"ContainerStarted","Data":"fbb4a49620ba05b72c686d86425e5ee48a21e375d5b5f806186a1c7702232d16"}
Jul 08 03:20:19.651489 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.651470    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:19.666737 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.666710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:19.694714 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:19.694669    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7" podStartSLOduration=9.053481749 podStartE2EDuration="29.694654662s" totalImagesPullingTime="20.641172913s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:19.694289131 +0000 UTC m=+30.802635071" watchObservedRunningTime="2026-07-08 03:20:19.694654662 +0000 UTC m=+30.803000626"
Jul 08 03:20:20.176759 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.176566    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5fdt"]
Jul 08 03:20:20.176874 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.176863    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:20.176990 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:20.176967    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:20.179337 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.179029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h6zkr"]
Jul 08 03:20:20.179337 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.179161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:20.179337 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:20.179255    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:20.179662 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.179647    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-skcm9"]
Jul 08 03:20:20.179727 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.179715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:20.179817 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:20.179800    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:20.655659 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.655626    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="617e06b1931136fda9469fed458bf7961064e58c1610c0cd9b53ec8e55fba2f7" exitCode=0
Jul 08 03:20:20.656104 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:20.655717    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"617e06b1931136fda9469fed458bf7961064e58c1610c0cd9b53ec8e55fba2f7"}
Jul 08 03:20:21.379555 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:21.379520    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:21.379713 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:21.379653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:21.379713 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:21.379687    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:21.379857 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:21.379752    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:22.379249 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:22.379181    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:22.379809 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:22.379314    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:23.109231 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:23.109199    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:23.109231 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:23.109247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:23.109484 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.109365    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:23.109484 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.109372    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:23.109484 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.109429    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:20:55.109412969 +0000 UTC m=+66.217758889 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 03:20:23.109484 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.109449    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret podName:496ab302-a93d-4c2b-96ba-7632f4c92c9c nodeName:}" failed. No retries permitted until 2026-07-08 03:20:55.109440388 +0000 UTC m=+66.217786310 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret") pod "global-pull-secret-syncer-h6zkr" (UID: "496ab302-a93d-4c2b-96ba-7632f4c92c9c") : object "kube-system"/"original-pull-secret" not registered
Jul 08 03:20:23.209832 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:23.209799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:23.209985 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.209948    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 03:20:23.209985 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.209966    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 03:20:23.209985 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.209975    2563 projected.go:196] Error preparing data for projected volume kube-api-access-rd2pw for pod openshift-network-diagnostics/network-check-target-skcm9: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:23.210173 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.210035    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw podName:4deb049a-2ca9-4877-9647-9ad5f5cc489e nodeName:}" failed. No retries permitted until 2026-07-08 03:20:55.210015894 +0000 UTC m=+66.318361832 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rd2pw" (UniqueName: "kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw") pod "network-check-target-skcm9" (UID: "4deb049a-2ca9-4877-9647-9ad5f5cc489e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 03:20:23.378953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:23.378870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:23.378953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:23.378923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:23.379227 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.379026    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:23.379227 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:23.379171    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:24.379301 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:24.379227    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:24.379694 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:24.379339    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h6zkr" podUID="496ab302-a93d-4c2b-96ba-7632f4c92c9c"
Jul 08 03:20:25.378957 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:25.378938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:25.379046 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:25.379020    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-skcm9" podUID="4deb049a-2ca9-4877-9647-9ad5f5cc489e"
Jul 08 03:20:25.379102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:25.379081    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:25.379249 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:25.379227    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:20:25.671267 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:25.671189    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="c606f2948104db71ade256013e1db9ec6ed955e0fd3b5e077aaeddfb082e49a6" exitCode=0
Jul 08 03:20:25.671267 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:25.671249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"c606f2948104db71ade256013e1db9ec6ed955e0fd3b5e077aaeddfb082e49a6"}
Jul 08 03:20:26.183959 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.183935    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-160.ec2.internal" event="NodeReady"
Jul 08 03:20:26.184097 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.184043    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 03:20:26.230944 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.230912    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9cxdf"]
Jul 08 03:20:26.250027 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.249990    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mbkbh"]
Jul 08 03:20:26.250183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.250110    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.252835 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.252810    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 03:20:26.253294 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.252885    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 03:20:26.253410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.253354    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gjrkj\""
Jul 08 03:20:26.273120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.273095    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9cxdf"]
Jul 08 03:20:26.273263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.273154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mbkbh"]
Jul 08 03:20:26.273263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.273216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.275453 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.275434    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 03:20:26.275453 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.275451    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 03:20:26.275604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.275525    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-v57gq\""
Jul 08 03:20:26.276512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.276280    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 03:20:26.337611 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.337584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d498c1-68f2-4915-8504-80e20964154d-config-volume\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.337786 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.337638    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp878\" (UniqueName: \"kubernetes.io/projected/a3d498c1-68f2-4915-8504-80e20964154d-kube-api-access-rp878\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.337786 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.337695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.337786 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.337711    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d498c1-68f2-4915-8504-80e20964154d-tmp-dir\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.378820 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.378786    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:26.381591 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.381574    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 03:20:26.438691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d19979df-55c8-4e1e-8a25-0390633d1794-cert\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.438691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp878\" (UniqueName: \"kubernetes.io/projected/a3d498c1-68f2-4915-8504-80e20964154d-kube-api-access-rp878\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.438889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.438889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438740    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d498c1-68f2-4915-8504-80e20964154d-tmp-dir\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.438889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438765    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbzlk\" (UniqueName: \"kubernetes.io/projected/d19979df-55c8-4e1e-8a25-0390633d1794-kube-api-access-vbzlk\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.438889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.438790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d498c1-68f2-4915-8504-80e20964154d-config-volume\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.438889 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:26.438868    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:26.439119 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:26.438944    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:26.93892315 +0000 UTC m=+38.047269071 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:26.439119 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.439052    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3d498c1-68f2-4915-8504-80e20964154d-tmp-dir\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.439287 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.439271    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a3d498c1-68f2-4915-8504-80e20964154d-config-volume\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.448762 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.448742    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp878\" (UniqueName: \"kubernetes.io/projected/a3d498c1-68f2-4915-8504-80e20964154d-kube-api-access-rp878\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.539965 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.539931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbzlk\" (UniqueName: \"kubernetes.io/projected/d19979df-55c8-4e1e-8a25-0390633d1794-kube-api-access-vbzlk\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.540118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.539981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d19979df-55c8-4e1e-8a25-0390633d1794-cert\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.542669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.542610    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d19979df-55c8-4e1e-8a25-0390633d1794-cert\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.549735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.549717    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbzlk\" (UniqueName: \"kubernetes.io/projected/d19979df-55c8-4e1e-8a25-0390633d1794-kube-api-access-vbzlk\") pod \"ingress-canary-mbkbh\" (UID: \"d19979df-55c8-4e1e-8a25-0390633d1794\") " pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.582823 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.582797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mbkbh"
Jul 08 03:20:26.677099 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.676923    2563 generic.go:356] "Generic (PLEG): container finished" podID="ba1ffb6a-2588-4411-b936-433be263812f" containerID="5bc9c5db4d50c6098876d7ac61bd4cf90902d1f230a58d871f4931d553308ead" exitCode=0
Jul 08 03:20:26.677099 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.676999    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerDied","Data":"5bc9c5db4d50c6098876d7ac61bd4cf90902d1f230a58d871f4931d553308ead"}
Jul 08 03:20:26.783789 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.783494    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mbkbh"]
Jul 08 03:20:26.788761 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:20:26.788734    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19979df_55c8_4e1e_8a25_0390633d1794.slice/crio-0e43df5ed4fd27ae0185a92517f86788875457a1cfd787cf02be89abbc24980a WatchSource:0}: Error finding container 0e43df5ed4fd27ae0185a92517f86788875457a1cfd787cf02be89abbc24980a: Status 404 returned error can't find the container with id 0e43df5ed4fd27ae0185a92517f86788875457a1cfd787cf02be89abbc24980a
Jul 08 03:20:26.944207 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:26.944169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:26.944372 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:26.944279    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:26.944372 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:26.944328    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:27.944315524 +0000 UTC m=+39.052661441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:27.378920 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.378886    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:27.379090 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.378896    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:27.381525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.381503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 03:20:27.381648 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.381553    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 03:20:27.381648 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.381575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb8ld\""
Jul 08 03:20:27.381747 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.381728    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2m54\""
Jul 08 03:20:27.381841 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.381824    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 03:20:27.680691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.680584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mbkbh" event={"ID":"d19979df-55c8-4e1e-8a25-0390633d1794","Type":"ContainerStarted","Data":"0e43df5ed4fd27ae0185a92517f86788875457a1cfd787cf02be89abbc24980a"}
Jul 08 03:20:27.684495 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.684471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" event={"ID":"ba1ffb6a-2588-4411-b936-433be263812f","Type":"ContainerStarted","Data":"59e33463e294e25cf480fdab11f46b526be21acf3b75546fbebdca5070177237"}
Jul 08 03:20:27.711311 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.711259    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wf9lr" podStartSLOduration=8.041009929 podStartE2EDuration="37.711238357s" totalImagesPullingTime="29.670228428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:27.710765172 +0000 UTC m=+38.819111106" watchObservedRunningTime="2026-07-08 03:20:27.711238357 +0000 UTC m=+38.819584296"
Jul 08 03:20:27.951673 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:27.951595    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:27.951837 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:27.951752    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:27.951837 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:27.951827    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:29.951806338 +0000 UTC m=+41.060152256 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:29.965809 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:29.965626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:29.965809 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:29.965788    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:29.966314 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:29.965853    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:33.965837272 +0000 UTC m=+45.074183190 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:30.692210 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:30.692174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mbkbh" event={"ID":"d19979df-55c8-4e1e-8a25-0390633d1794","Type":"ContainerStarted","Data":"080f40e335d9fd1bb7755319159fb492f02319d74116957280a34d3d44f0cf13"}
Jul 08 03:20:30.707069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:30.707020    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mbkbh" podStartSLOduration=1.8761150880000002 podStartE2EDuration="4.707007285s" totalImagesPullingTime="2.830892197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:30.7067994 +0000 UTC m=+41.815145340" watchObservedRunningTime="2026-07-08 03:20:30.707007285 +0000 UTC m=+41.815353225"
Jul 08 03:20:33.997015 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:33.996980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:33.997489 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:33.997090    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:33.997489 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:33.997162    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:41.997128281 +0000 UTC m=+53.105474198 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:42.051073 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:42.051029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:42.051554 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:42.051169    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:42.051554 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:42.051233    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:20:58.051218322 +0000 UTC m=+69.159564239 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:51.668821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:51.668793    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-h5lx7"
Jul 08 03:20:55.144132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.144083    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:55.144531 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.144163    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:20:55.146398 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.146381    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 03:20:55.146443 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.146397    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 03:20:55.155262 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:55.155244    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:20:55.155315 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:55.155304    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:21:59.155284672 +0000 UTC m=+130.263630590 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : secret "metrics-daemon-secret" not found
Jul 08 03:20:55.158971 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.158955    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/496ab302-a93d-4c2b-96ba-7632f4c92c9c-original-pull-secret\") pod \"global-pull-secret-syncer-h6zkr\" (UID: \"496ab302-a93d-4c2b-96ba-7632f4c92c9c\") " pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:55.187051 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.187028    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h6zkr"
Jul 08 03:20:55.246159 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.246107    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:55.252656 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.252454    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 03:20:55.265581 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.264540    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 03:20:55.276091 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.276061    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd2pw\" (UniqueName: \"kubernetes.io/projected/4deb049a-2ca9-4877-9647-9ad5f5cc489e-kube-api-access-rd2pw\") pod \"network-check-target-skcm9\" (UID: \"4deb049a-2ca9-4877-9647-9ad5f5cc489e\") " pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:55.295021 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.294999    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-x2m54\""
Jul 08 03:20:55.303703 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.303685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:20:55.321447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.321422    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h6zkr"]
Jul 08 03:20:55.324699 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:20:55.324673    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod496ab302_a93d_4c2b_96ba_7632f4c92c9c.slice/crio-483960562fd75b4f95b6cf77d6a799955bc27adf114c395338c791f192927204 WatchSource:0}: Error finding container 483960562fd75b4f95b6cf77d6a799955bc27adf114c395338c791f192927204: Status 404 returned error can't find the container with id 483960562fd75b4f95b6cf77d6a799955bc27adf114c395338c791f192927204
Jul 08 03:20:55.422166 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.422078    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-skcm9"]
Jul 08 03:20:55.425445 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:20:55.425424    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4deb049a_2ca9_4877_9647_9ad5f5cc489e.slice/crio-1d61d7ad20a6f92d5437f12e9335cc2e81e8434e5e76fc74a0e693cb834e6111 WatchSource:0}: Error finding container 1d61d7ad20a6f92d5437f12e9335cc2e81e8434e5e76fc74a0e693cb834e6111: Status 404 returned error can't find the container with id 1d61d7ad20a6f92d5437f12e9335cc2e81e8434e5e76fc74a0e693cb834e6111
Jul 08 03:20:55.754101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.754007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h6zkr" event={"ID":"496ab302-a93d-4c2b-96ba-7632f4c92c9c","Type":"ContainerStarted","Data":"483960562fd75b4f95b6cf77d6a799955bc27adf114c395338c791f192927204"}
Jul 08 03:20:55.754986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:55.754964    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-skcm9" event={"ID":"4deb049a-2ca9-4877-9647-9ad5f5cc489e","Type":"ContainerStarted","Data":"1d61d7ad20a6f92d5437f12e9335cc2e81e8434e5e76fc74a0e693cb834e6111"}
Jul 08 03:20:58.069189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:58.069132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:20:58.069662 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:58.069308    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:20:58.069662 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:20:58.069385    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:21:30.069363846 +0000 UTC m=+101.177709768 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:20:59.767169 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:59.767120    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-skcm9" event={"ID":"4deb049a-2ca9-4877-9647-9ad5f5cc489e","Type":"ContainerStarted","Data":"aea22c4f9bc815e164b914c72224d81a9d8b3f928b19f95ee20eea3f43e514ea"}
Jul 08 03:20:59.767604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:20:59.767285    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:21:00.771026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:00.770944    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h6zkr" event={"ID":"496ab302-a93d-4c2b-96ba-7632f4c92c9c","Type":"ContainerStarted","Data":"41e05dd946a0b85142ac348cfc88fc712b326ba77755534d3c46b38457cc7e09"}
Jul 08 03:21:00.789451 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:00.786347    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-skcm9" podStartSLOduration=67.384371273 podStartE2EDuration="1m10.786331856s" totalImagesPullingTime="3.401960583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:20:59.783279757 +0000 UTC m=+70.891625699" watchObservedRunningTime="2026-07-08 03:21:00.786331856 +0000 UTC m=+71.894677796"
Jul 08 03:21:00.789451 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:00.787750    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h6zkr" podStartSLOduration=65.742058782 podStartE2EDuration="1m10.787730061s" totalImagesPullingTime="5.045671279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:21:00.784949896 +0000 UTC m=+71.893295837" watchObservedRunningTime="2026-07-08 03:21:00.787730061 +0000 UTC m=+71.896076005"
Jul 08 03:21:23.927010 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.926896    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"]
Jul 08 03:21:23.930937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.930921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:23.933182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.933158    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vnwfn\""
Jul 08 03:21:23.933280 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.933183    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 03:21:23.933956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.933938    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 08 03:21:23.934047 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.933959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 03:21:23.934047 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.933996    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 03:21:23.936953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:23.936933    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"]
Jul 08 03:21:24.053025 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.052986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v9jg\" (UniqueName: \"kubernetes.io/projected/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-kube-api-access-8v9jg\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.053243 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.053066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.153395 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.153357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.153395 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.153399    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v9jg\" (UniqueName: \"kubernetes.io/projected/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-kube-api-access-8v9jg\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.155961 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.155931    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.162568 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.162547    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v9jg\" (UniqueName: \"kubernetes.io/projected/3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13-kube-api-access-8v9jg\") pod \"managed-serviceaccount-addon-agent-768d5bc658-swxsm\" (UID: \"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.247988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.247900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"
Jul 08 03:21:24.370422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.370387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm"]
Jul 08 03:21:24.376786 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:21:24.376761    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d74d750_9f5f_49d4_b4c0_81a0f8c0dc13.slice/crio-fd8d9ea674338d4f80c547d8e052544921dd3cea342f53e82dfc86464b164144 WatchSource:0}: Error finding container fd8d9ea674338d4f80c547d8e052544921dd3cea342f53e82dfc86464b164144: Status 404 returned error can't find the container with id fd8d9ea674338d4f80c547d8e052544921dd3cea342f53e82dfc86464b164144
Jul 08 03:21:24.831566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:24.831533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm" event={"ID":"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13","Type":"ContainerStarted","Data":"fd8d9ea674338d4f80c547d8e052544921dd3cea342f53e82dfc86464b164144"}
Jul 08 03:21:27.840724 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:27.840690    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm" event={"ID":"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13","Type":"ContainerStarted","Data":"588d702ddee1511298b6685e5cbcfd16bb3a2e7e04a4709e037b71abfb618991"}
Jul 08 03:21:27.856984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:27.856940    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm" podStartSLOduration=1.695679745 podStartE2EDuration="4.856927332s" totalImagesPullingTime="3.161247587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:21:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:21:27.856367123 +0000 UTC m=+98.964713090" watchObservedRunningTime="2026-07-08 03:21:27.856927332 +0000 UTC m=+98.965273311"
Jul 08 03:21:30.091618 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:30.091577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:21:30.092178 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:21:30.091739    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:21:30.092178 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:21:30.091821    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:22:34.091796615 +0000 UTC m=+165.200142533 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:21:30.773274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:30.773245    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-skcm9"
Jul 08 03:21:59.188577 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:21:59.188540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:21:59.189202 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:21:59.188730    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:21:59.189202 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:21:59.188832    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:01.188810851 +0000 UTC m=+252.297156773 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : secret "metrics-daemon-secret" not found
Jul 08 03:22:29.260472 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:22:29.260434    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9cxdf" podUID="a3d498c1-68f2-4915-8504-80e20964154d"
Jul 08 03:22:29.991298 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:22:29.991270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:22:30.389208 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:22:30.389174    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:22:34.104306 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:22:34.104261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:22:34.104707 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:22:34.104372    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 03:22:34.104707 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:22:34.104424    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls podName:a3d498c1-68f2-4915-8504-80e20964154d nodeName:}" failed. No retries permitted until 2026-07-08 03:24:36.1044084 +0000 UTC m=+287.212754318 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls") pod "dns-default-9cxdf" (UID: "a3d498c1-68f2-4915-8504-80e20964154d") : secret "dns-default-metrics-tls" not found
Jul 08 03:22:41.379316 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:22:41.379230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:23:28.126897 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:23:28.126863    2563 generic.go:356] "Generic (PLEG): container finished" podID="3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13" containerID="588d702ddee1511298b6685e5cbcfd16bb3a2e7e04a4709e037b71abfb618991" exitCode=255
Jul 08 03:23:28.127344 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:23:28.126918    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm" event={"ID":"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13","Type":"ContainerDied","Data":"588d702ddee1511298b6685e5cbcfd16bb3a2e7e04a4709e037b71abfb618991"}
Jul 08 03:23:28.132354 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:23:28.132333    2563 scope.go:117] "RemoveContainer" containerID="588d702ddee1511298b6685e5cbcfd16bb3a2e7e04a4709e037b71abfb618991"
Jul 08 03:23:29.134151 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:23:29.134114    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-768d5bc658-swxsm" event={"ID":"3d74d750-9f5f-49d4-b4c0-81a0f8c0dc13","Type":"ContainerStarted","Data":"3f89eb6edd98aa26e04284256641875c908f9f6bb0a52b8d31ae5f324321a887"}
Jul 08 03:24:01.282291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:01.282248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:24:01.282795 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:01.282390    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 03:24:01.282795 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:01.282448    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs podName:848ea393-51f5-40f4-919f-7b147f20ff22 nodeName:}" failed. No retries permitted until 2026-07-08 03:26:03.282433668 +0000 UTC m=+374.390779586 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs") pod "network-metrics-daemon-v5fdt" (UID: "848ea393-51f5-40f4-919f-7b147f20ff22") : secret "metrics-daemon-secret" not found
Jul 08 03:24:13.313101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.313004    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:24:13.315859 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.315842    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-cf9c66499-8vtcz"]
Jul 08 03:24:13.315995 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.315975    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.318339 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.318316    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 03:24:13.318448 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.318347    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 03:24:13.318511 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.318477    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5t5kv\""
Jul 08 03:24:13.318566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.318510    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 03:24:13.318566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.318543    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.321655 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.321634    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 08 03:24:13.322439 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-7wr4z\""
Jul 08 03:24:13.322542 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322472    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 08 03:24:13.322542 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322535    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 08 03:24:13.322724 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322706    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.322835 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322807    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 08 03:24:13.322910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.322808    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.326041 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.326025    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 03:24:13.331501 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.331482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cf9c66499-8vtcz"]
Jul 08 03:24:13.335210 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.335190    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:24:13.408026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.407998    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"]
Jul 08 03:24:13.410884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.410867    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-9lstn"]
Jul 08 03:24:13.410991 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.410974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.413165 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413130    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 08 03:24:13.413475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413451    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 08 03:24:13.413591 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413563    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.413650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413631    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6rvp"]
Jul 08 03:24:13.413749 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.414014 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413851    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pnpl8\""
Jul 08 03:24:13.414014 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.413913    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.416342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416326    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.416436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 08 03:24:13.416535 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 08 03:24:13.416576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.416576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416559    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-9mx2s\""
Jul 08 03:24:13.416674 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.416549    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.419089 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.419063    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 03:24:13.419215 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.419121    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 08 03:24:13.419470 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.419445    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 08 03:24:13.419563 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.419498    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7s2bz\""
Jul 08 03:24:13.419694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.419676    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 03:24:13.422622 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.422591    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 08 03:24:13.423614 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.423597    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"]
Jul 08 03:24:13.424543 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.424514    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-9lstn"]
Jul 08 03:24:13.425643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.425625    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 08 03:24:13.438829 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.438806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6rvp"]
Jul 08 03:24:13.465701 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-stats-auth\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.465701 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8qkr\" (UniqueName: \"kubernetes.io/projected/aff7f08a-06ac-41bb-9e76-14f8670b4c84-kube-api-access-j8qkr\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465771    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465796    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.465973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpgvg\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466338 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.466022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-default-certificate\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.466338 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.466072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.466338 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.466098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.567291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.567291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf02e941-4d09-4510-86f4-7035ff6ff81c-config\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.567291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-serving-cert\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567322    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-config\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567339    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf02e941-4d09-4510-86f4-7035ff6ff81c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-742t4\" (UniqueName: \"kubernetes.io/projected/cf02e941-4d09-4510-86f4-7035ff6ff81c-kube-api-access-742t4\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567379    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567411    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/788ca259-379f-4dfb-9f16-9fc1f443ecd8-serving-cert\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567514    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.567473    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 03:24:13.567558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567542    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cpzl\" (UniqueName: \"kubernetes.io/projected/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-kube-api-access-5cpzl\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpgvg\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.567605    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:14.067584389 +0000 UTC m=+265.175930310 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : secret "router-metrics-certs-default" not found
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-default-certificate\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567675    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-tmp\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-stats-auth\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567769    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-snapshots\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567797    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8qkr\" (UniqueName: \"kubernetes.io/projected/aff7f08a-06ac-41bb-9e76-14f8670b4c84-kube-api-access-j8qkr\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567849    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-trusted-ca\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.568069 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.567905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdtx2\" (UniqueName: \"kubernetes.io/projected/788ca259-379f-4dfb-9f16-9fc1f443ecd8-kube-api-access-bdtx2\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.567954    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:14.067936883 +0000 UTC m=+265.176282804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.568037    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.568040    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.568371    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.568387    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67499fc746-fm5gv: secret "image-registry-tls" not found
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:13.568440    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls podName:5f564f08-1db0-496d-b77d-2a7d1cc98203 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:14.068422467 +0000 UTC m=+265.176768398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls") pod "image-registry-67499fc746-fm5gv" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203") : secret "image-registry-tls" not found
Jul 08 03:24:13.569042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.568820    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.570238 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.570214    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.570567 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.570548    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-stats-auth\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.570695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.570674    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.570752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.570711    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-default-certificate\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.575680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.575649    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.576336 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.576315    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpgvg\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:13.576441 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.576403    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8qkr\" (UniqueName: \"kubernetes.io/projected/aff7f08a-06ac-41bb-9e76-14f8670b4c84-kube-api-access-j8qkr\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:13.669326 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-serving-cert\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669326 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-config\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf02e941-4d09-4510-86f4-7035ff6ff81c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-742t4\" (UniqueName: \"kubernetes.io/projected/cf02e941-4d09-4510-86f4-7035ff6ff81c-kube-api-access-742t4\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/788ca259-379f-4dfb-9f16-9fc1f443ecd8-serving-cert\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cpzl\" (UniqueName: \"kubernetes.io/projected/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-kube-api-access-5cpzl\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669456    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-tmp\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669711    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-snapshots\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.669878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-trusted-ca\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.669878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdtx2\" (UniqueName: \"kubernetes.io/projected/788ca259-379f-4dfb-9f16-9fc1f443ecd8-kube-api-access-bdtx2\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.669878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.669848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf02e941-4d09-4510-86f4-7035ff6ff81c-config\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.670102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670076    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.670653 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670300    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-config\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.670653 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670598    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cf02e941-4d09-4510-86f4-7035ff6ff81c-config\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.670828 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670704    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-tmp\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.670958 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670875    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-snapshots\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.670958 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670904    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.671066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.670990    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/788ca259-379f-4dfb-9f16-9fc1f443ecd8-trusted-ca\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.671883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.671865    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cf02e941-4d09-4510-86f4-7035ff6ff81c-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.672072 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.672054    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-serving-cert\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.672246 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.672228    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/788ca259-379f-4dfb-9f16-9fc1f443ecd8-serving-cert\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.678189 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.678165    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdtx2\" (UniqueName: \"kubernetes.io/projected/788ca259-379f-4dfb-9f16-9fc1f443ecd8-kube-api-access-bdtx2\") pod \"console-operator-68b757fb67-9lstn\" (UID: \"788ca259-379f-4dfb-9f16-9fc1f443ecd8\") " pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.678365 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.678349    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-742t4\" (UniqueName: \"kubernetes.io/projected/cf02e941-4d09-4510-86f4-7035ff6ff81c-kube-api-access-742t4\") pod \"service-ca-operator-5d46d9ffd4-6cs8h\" (UID: \"cf02e941-4d09-4510-86f4-7035ff6ff81c\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.678723 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.678702    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cpzl\" (UniqueName: \"kubernetes.io/projected/2f8cdf2b-e0ba-4223-aa42-431ec6ac0814-kube-api-access-5cpzl\") pod \"insights-operator-5c847dcfc4-r6rvp\" (UID: \"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814\") " pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.722937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.722905    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"
Jul 08 03:24:13.729168 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.729152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:13.734926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.734906    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp"
Jul 08 03:24:13.869766 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.869621    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-9lstn"]
Jul 08 03:24:13.872779 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:13.872747    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod788ca259_379f_4dfb_9f16_9fc1f443ecd8.slice/crio-b6b2d2f652ed8639597dcfc6b760deb921ffdae46b2bdda36d15d8b2c327698a WatchSource:0}: Error finding container b6b2d2f652ed8639597dcfc6b760deb921ffdae46b2bdda36d15d8b2c327698a: Status 404 returned error can't find the container with id b6b2d2f652ed8639597dcfc6b760deb921ffdae46b2bdda36d15d8b2c327698a
Jul 08 03:24:13.879412 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.879387    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h"]
Jul 08 03:24:13.882242 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:13.882212    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf02e941_4d09_4510_86f4_7035ff6ff81c.slice/crio-4defbec8c03f94ed1fb32061d017110855e1252af3ae0c370a7bf61c38ee4ead WatchSource:0}: Error finding container 4defbec8c03f94ed1fb32061d017110855e1252af3ae0c370a7bf61c38ee4ead: Status 404 returned error can't find the container with id 4defbec8c03f94ed1fb32061d017110855e1252af3ae0c370a7bf61c38ee4ead
Jul 08 03:24:13.901903 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:13.899824    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-r6rvp"]
Jul 08 03:24:13.902823 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:13.902799    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f8cdf2b_e0ba_4223_aa42_431ec6ac0814.slice/crio-f6dca06b2c7aa1e0e2dabfead33d07f6599a05c686be672ee36bffd0778016d5 WatchSource:0}: Error finding container f6dca06b2c7aa1e0e2dabfead33d07f6599a05c686be672ee36bffd0778016d5: Status 404 returned error can't find the container with id f6dca06b2c7aa1e0e2dabfead33d07f6599a05c686be672ee36bffd0778016d5
Jul 08 03:24:14.073529 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.073442    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:14.073529 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.073482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:14.073529 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.073519    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073596    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073615    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073632    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67499fc746-fm5gv: secret "image-registry-tls" not found
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:15.073613002 +0000 UTC m=+266.181958927 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073662    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:15.073651628 +0000 UTC m=+266.181997546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : secret "router-metrics-certs-default" not found
Jul 08 03:24:14.073810 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:14.073681    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls podName:5f564f08-1db0-496d-b77d-2a7d1cc98203 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:15.073668463 +0000 UTC m=+266.182014416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls") pod "image-registry-67499fc746-fm5gv" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203") : secret "image-registry-tls" not found
Jul 08 03:24:14.257474 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.257438    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" event={"ID":"788ca259-379f-4dfb-9f16-9fc1f443ecd8","Type":"ContainerStarted","Data":"b6b2d2f652ed8639597dcfc6b760deb921ffdae46b2bdda36d15d8b2c327698a"}
Jul 08 03:24:14.258509 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.258483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h" event={"ID":"cf02e941-4d09-4510-86f4-7035ff6ff81c","Type":"ContainerStarted","Data":"4defbec8c03f94ed1fb32061d017110855e1252af3ae0c370a7bf61c38ee4ead"}
Jul 08 03:24:14.259422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:14.259400    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp" event={"ID":"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814","Type":"ContainerStarted","Data":"f6dca06b2c7aa1e0e2dabfead33d07f6599a05c686be672ee36bffd0778016d5"}
Jul 08 03:24:15.082994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:15.082953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:15.083020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:15.083075    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083240    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083258    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083273    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67499fc746-fm5gv: secret "image-registry-tls" not found
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083297    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:17.083279337 +0000 UTC m=+268.191625259 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : secret "router-metrics-certs-default" not found
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083315    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:17.083305643 +0000 UTC m=+268.191651562 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:15.083511 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:15.083330    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls podName:5f564f08-1db0-496d-b77d-2a7d1cc98203 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:17.083322003 +0000 UTC m=+268.191667921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls") pod "image-registry-67499fc746-fm5gv" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203") : secret "image-registry-tls" not found
Jul 08 03:24:17.102644 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.102608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.102692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102739    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102757    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67499fc746-fm5gv: secret "image-registry-tls" not found
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102800    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls podName:5f564f08-1db0-496d-b77d-2a7d1cc98203 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.102785626 +0000 UTC m=+272.211131549 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls") pod "image-registry-67499fc746-fm5gv" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203") : secret "image-registry-tls" not found
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.102755    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102837    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102844    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.102828737 +0000 UTC m=+272.211174675 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:17.103042 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:17.102890    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.102875726 +0000 UTC m=+272.211221659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : secret "router-metrics-certs-default" not found
Jul 08 03:24:17.271432 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.271164    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp" event={"ID":"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814","Type":"ContainerStarted","Data":"2e57a4c95ba3202048a078e9a2ab186ddd0a57944248398e03c9942c1034c5f4"}
Jul 08 03:24:17.272819 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.272795    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/0.log"
Jul 08 03:24:17.272942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.272840    2563 generic.go:356] "Generic (PLEG): container finished" podID="788ca259-379f-4dfb-9f16-9fc1f443ecd8" containerID="15397dfa8e3435a92f61e976a72069361eb4f345028759a7e481224d17f17653" exitCode=255
Jul 08 03:24:17.272942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.272925    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" event={"ID":"788ca259-379f-4dfb-9f16-9fc1f443ecd8","Type":"ContainerDied","Data":"15397dfa8e3435a92f61e976a72069361eb4f345028759a7e481224d17f17653"}
Jul 08 03:24:17.273105 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.273086    2563 scope.go:117] "RemoveContainer" containerID="15397dfa8e3435a92f61e976a72069361eb4f345028759a7e481224d17f17653"
Jul 08 03:24:17.274520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.274497    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h" event={"ID":"cf02e941-4d09-4510-86f4-7035ff6ff81c","Type":"ContainerStarted","Data":"5dfdcc0e24343905c02b18d0ce0ba5b9473b3dee94381bbdb0fd3bb73dedf180"}
Jul 08 03:24:17.289857 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.289512    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp" podStartSLOduration=1.098524884 podStartE2EDuration="4.289495464s" totalImagesPullingTime="3.19097058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:17.288589666 +0000 UTC m=+268.396935606" watchObservedRunningTime="2026-07-08 03:24:17.289495464 +0000 UTC m=+268.397841406"
Jul 08 03:24:17.320578 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:17.320502    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h" podStartSLOduration=1.105517955 podStartE2EDuration="4.320485798s" totalImagesPullingTime="3.214967843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:17.319642993 +0000 UTC m=+268.427988935" watchObservedRunningTime="2026-07-08 03:24:17.320485798 +0000 UTC m=+268.428831739"
Jul 08 03:24:18.215262 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.215231    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mfqgx"]
Jul 08 03:24:18.221502 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.221411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.224010 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.223985    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 03:24:18.224131 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.224017    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 03:24:18.224131 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.224060    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-kfwlq\""
Jul 08 03:24:18.230049 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.230029    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mfqgx"]
Jul 08 03:24:18.279116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279090    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:24:18.279500 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/0.log"
Jul 08 03:24:18.279570 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279520    2563 generic.go:356] "Generic (PLEG): container finished" podID="788ca259-379f-4dfb-9f16-9fc1f443ecd8" containerID="4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb" exitCode=255
Jul 08 03:24:18.279619 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279593    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" event={"ID":"788ca259-379f-4dfb-9f16-9fc1f443ecd8","Type":"ContainerDied","Data":"4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb"}
Jul 08 03:24:18.279672 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279661    2563 scope.go:117] "RemoveContainer" containerID="15397dfa8e3435a92f61e976a72069361eb4f345028759a7e481224d17f17653"
Jul 08 03:24:18.279958 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.279929    2563 scope.go:117] "RemoveContainer" containerID="4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb"
Jul 08 03:24:18.280181 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.280131    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-9lstn_openshift-console-operator(788ca259-379f-4dfb-9f16-9fc1f443ecd8)\"" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" podUID="788ca259-379f-4dfb-9f16-9fc1f443ecd8"
Jul 08 03:24:18.313740 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.313714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/047a2caa-fd23-46fb-9fbe-91cadb1a5788-data-volume\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.313974 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.313913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.314074 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.313979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/047a2caa-fd23-46fb-9fbe-91cadb1a5788-crio-socket\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.314192 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.314169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.314252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.314226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbrgv\" (UniqueName: \"kubernetes.io/projected/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-api-access-bbrgv\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415369 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415292    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415369 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbrgv\" (UniqueName: \"kubernetes.io/projected/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-api-access-bbrgv\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/047a2caa-fd23-46fb-9fbe-91cadb1a5788-data-volume\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.415440    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/047a2caa-fd23-46fb-9fbe-91cadb1a5788-crio-socket\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.415501    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:18.915483419 +0000 UTC m=+270.023829352 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.415505    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.415537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415534    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/047a2caa-fd23-46fb-9fbe-91cadb1a5788-crio-socket\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.415797 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.415578    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:18.91555826 +0000 UTC m=+270.023904198 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.415797 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.415747    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/047a2caa-fd23-46fb-9fbe-91cadb1a5788-data-volume\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.425551 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.425527    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbrgv\" (UniqueName: \"kubernetes.io/projected/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-api-access-bbrgv\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.919514 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.919481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.919803 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:18.919559    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:18.919803 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.919601    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.919803 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.919679    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:19.91966209 +0000 UTC m=+271.028008008 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:18.919803 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.919693    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:18.919803 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:18.919758    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:19.919740952 +0000 UTC m=+271.028086883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:19.283654 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:19.283579    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:24:19.283994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:19.283912    2563 scope.go:117] "RemoveContainer" containerID="4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb"
Jul 08 03:24:19.284086 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:19.284067    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-9lstn_openshift-console-operator(788ca259-379f-4dfb-9f16-9fc1f443ecd8)\"" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" podUID="788ca259-379f-4dfb-9f16-9fc1f443ecd8"
Jul 08 03:24:19.928058 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:19.928016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:19.928251 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:19.928108    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:19.928251 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:19.928166    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:19.928251 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:19.928232    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.928214773 +0000 UTC m=+273.036560691 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:19.928363 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:19.928262    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:19.928363 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:19.928314    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:21.928300365 +0000 UTC m=+273.036646284 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:20.786727 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:20.786701    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ffj7r_9478073e-2c45-4a5f-838f-ae62733ea6ee/dns-node-resolver/0.log"
Jul 08 03:24:21.137408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.137376    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:21.137566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.137414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:21.137566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.137462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:21.137566 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137538    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:29.137520243 +0000 UTC m=+280.245866161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:21.137677 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137573    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 03:24:21.137677 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137580    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 03:24:21.137677 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137598    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67499fc746-fm5gv: secret "image-registry-tls" not found
Jul 08 03:24:21.137677 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137619    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:29.137607235 +0000 UTC m=+280.245953158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : secret "router-metrics-certs-default" not found
Jul 08 03:24:21.137677 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.137637    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls podName:5f564f08-1db0-496d-b77d-2a7d1cc98203 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:29.137625271 +0000 UTC m=+280.245971221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls") pod "image-registry-67499fc746-fm5gv" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203") : secret "image-registry-tls" not found
Jul 08 03:24:21.487416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.487341    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rlrnb"]
Jul 08 03:24:21.491612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.491597    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.493963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.493931    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 08 03:24:21.494067 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.494039    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-lsn66\""
Jul 08 03:24:21.494239 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.494220    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 08 03:24:21.494928 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.494909    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 08 03:24:21.495008 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.494940    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 08 03:24:21.501797 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.501766    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rlrnb"]
Jul 08 03:24:21.541563 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.541537    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqwpc\" (UniqueName: \"kubernetes.io/projected/2bf68677-c314-4794-b2db-519705d4dccf-kube-api-access-lqwpc\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.541703 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.541577    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2bf68677-c314-4794-b2db-519705d4dccf-signing-key\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.541703 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.541671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2bf68677-c314-4794-b2db-519705d4dccf-signing-cabundle\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.642634 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.642597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqwpc\" (UniqueName: \"kubernetes.io/projected/2bf68677-c314-4794-b2db-519705d4dccf-kube-api-access-lqwpc\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.642785 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.642649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2bf68677-c314-4794-b2db-519705d4dccf-signing-key\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.642785 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.642701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2bf68677-c314-4794-b2db-519705d4dccf-signing-cabundle\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.643375 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.643348    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/2bf68677-c314-4794-b2db-519705d4dccf-signing-cabundle\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.645320 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.645301    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/2bf68677-c314-4794-b2db-519705d4dccf-signing-key\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.651935 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.651907    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqwpc\" (UniqueName: \"kubernetes.io/projected/2bf68677-c314-4794-b2db-519705d4dccf-kube-api-access-lqwpc\") pod \"service-ca-758f599c5-rlrnb\" (UID: \"2bf68677-c314-4794-b2db-519705d4dccf\") " pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.802960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.802930    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-rlrnb"
Jul 08 03:24:21.927392 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.927366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rlrnb"]
Jul 08 03:24:21.931778 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:21.931746    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bf68677_c314_4794_b2db_519705d4dccf.slice/crio-91dc270f6e52203bf7f374e47adb39e27fb4eaa59a37e4b73c101fabe4b8fc78 WatchSource:0}: Error finding container 91dc270f6e52203bf7f374e47adb39e27fb4eaa59a37e4b73c101fabe4b8fc78: Status 404 returned error can't find the container with id 91dc270f6e52203bf7f374e47adb39e27fb4eaa59a37e4b73c101fabe4b8fc78
Jul 08 03:24:21.945411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.945390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:21.945486 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.945447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:21.945520 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.945499    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:21.945559 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.945536    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:21.945559 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.945557    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:25.945540969 +0000 UTC m=+277.053886887 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:21.945639 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:21.945581    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:25.945567519 +0000 UTC m=+277.053913438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:21.989643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:21.989624    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpqlc_33643870-0989-4f93-9e49-61cd14adfd7f/node-ca/0.log"
Jul 08 03:24:22.294504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:22.294468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-rlrnb" event={"ID":"2bf68677-c314-4794-b2db-519705d4dccf","Type":"ContainerStarted","Data":"f69d20886e9fd01c23b514a9e01c631f454823bd1c95e90d8dfa63c7b0860247"}
Jul 08 03:24:22.294661 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:22.294525    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-rlrnb" event={"ID":"2bf68677-c314-4794-b2db-519705d4dccf","Type":"ContainerStarted","Data":"91dc270f6e52203bf7f374e47adb39e27fb4eaa59a37e4b73c101fabe4b8fc78"}
Jul 08 03:24:22.310971 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:22.310913    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-rlrnb" podStartSLOduration=1.310899423 podStartE2EDuration="1.310899423s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:22.309815452 +0000 UTC m=+273.418161394" watchObservedRunningTime="2026-07-08 03:24:22.310899423 +0000 UTC m=+273.419245364"
Jul 08 03:24:22.787840 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:22.787770    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mbkbh_d19979df-55c8-4e1e-8a25-0390633d1794/serve-healthcheck-canary/0.log"
Jul 08 03:24:23.729974 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:23.729890    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:23.729974 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:23.729965    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:23.730493 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:23.730340    2563 scope.go:117] "RemoveContainer" containerID="4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb"
Jul 08 03:24:23.730548 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:23.730495    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-9lstn_openshift-console-operator(788ca259-379f-4dfb-9f16-9fc1f443ecd8)\"" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" podUID="788ca259-379f-4dfb-9f16-9fc1f443ecd8"
Jul 08 03:24:25.980516 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:25.980482    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:25.980890 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:25.980577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:25.980890 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:25.980645    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 03:24:25.980890 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:25.980643    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:25.980890 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:25.980705    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:33.980688698 +0000 UTC m=+285.089034616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : secret "insights-runtime-extractor-tls" not found
Jul 08 03:24:25.980890 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:25.980719    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm podName:047a2caa-fd23-46fb-9fbe-91cadb1a5788 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:33.980712639 +0000 UTC m=+285.089058557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mfqgx" (UID: "047a2caa-fd23-46fb-9fbe-91cadb1a5788") : configmap "kube-rbac-proxy" not found
Jul 08 03:24:29.208394 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.208359    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:29.208817 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.208439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:29.208817 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.208474    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:29.208817 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:29.208622    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle podName:aff7f08a-06ac-41bb-9e76-14f8670b4c84 nodeName:}" failed. No retries permitted until 2026-07-08 03:24:45.208603134 +0000 UTC m=+296.316949055 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle") pod "router-default-cf9c66499-8vtcz" (UID: "aff7f08a-06ac-41bb-9e76-14f8670b4c84") : configmap references non-existent config key: service-ca.crt
Jul 08 03:24:29.210978 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.210957    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/aff7f08a-06ac-41bb-9e76-14f8670b4c84-metrics-certs\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:29.211052 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.211023    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"image-registry-67499fc746-fm5gv\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") " pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:29.227252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.227231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:29.355970 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:29.355946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:24:29.359691 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:29.359669    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f564f08_1db0_496d_b77d_2a7d1cc98203.slice/crio-d280b8ef2c13361f17b1dfd60de08cb4dac00372dbf5ff40010446bbe61553f6 WatchSource:0}: Error finding container d280b8ef2c13361f17b1dfd60de08cb4dac00372dbf5ff40010446bbe61553f6: Status 404 returned error can't find the container with id d280b8ef2c13361f17b1dfd60de08cb4dac00372dbf5ff40010446bbe61553f6
Jul 08 03:24:30.319852 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:30.319819    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" event={"ID":"5f564f08-1db0-496d-b77d-2a7d1cc98203","Type":"ContainerStarted","Data":"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"}
Jul 08 03:24:30.320238 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:30.319872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" event={"ID":"5f564f08-1db0-496d-b77d-2a7d1cc98203","Type":"ContainerStarted","Data":"d280b8ef2c13361f17b1dfd60de08cb4dac00372dbf5ff40010446bbe61553f6"}
Jul 08 03:24:30.320238 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:30.319900    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:30.341628 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:30.341586    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" podStartSLOduration=17.341572218 podStartE2EDuration="17.341572218s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:30.341179735 +0000 UTC m=+281.449525672" watchObservedRunningTime="2026-07-08 03:24:30.341572218 +0000 UTC m=+281.449918157"
Jul 08 03:24:32.991974 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:32.991930    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9cxdf" podUID="a3d498c1-68f2-4915-8504-80e20964154d"
Jul 08 03:24:33.329398 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:33.329370    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:34.046564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.046527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:34.047035 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.046600    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:34.047106 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.047066    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/047a2caa-fd23-46fb-9fbe-91cadb1a5788-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:34.049021 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.048996    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/047a2caa-fd23-46fb-9fbe-91cadb1a5788-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mfqgx\" (UID: \"047a2caa-fd23-46fb-9fbe-91cadb1a5788\") " pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:34.131509 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.131480    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mfqgx"
Jul 08 03:24:34.281475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.281333    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mfqgx"]
Jul 08 03:24:34.283458 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:34.283431    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod047a2caa_fd23_46fb_9fbe_91cadb1a5788.slice/crio-e7ce1c476151c14f58b7dc50fdff37fe16662759d4be95913952484853e3d6da WatchSource:0}: Error finding container e7ce1c476151c14f58b7dc50fdff37fe16662759d4be95913952484853e3d6da: Status 404 returned error can't find the container with id e7ce1c476151c14f58b7dc50fdff37fe16662759d4be95913952484853e3d6da
Jul 08 03:24:34.333035 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:34.332943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfqgx" event={"ID":"047a2caa-fd23-46fb-9fbe-91cadb1a5788","Type":"ContainerStarted","Data":"e7ce1c476151c14f58b7dc50fdff37fe16662759d4be95913952484853e3d6da"}
Jul 08 03:24:36.164018 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:36.163973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:36.166677 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:36.166649    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a3d498c1-68f2-4915-8504-80e20964154d-metrics-tls\") pod \"dns-default-9cxdf\" (UID: \"a3d498c1-68f2-4915-8504-80e20964154d\") " pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:36.332407 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:36.332370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gjrkj\""
Jul 08 03:24:36.340120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:36.340084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:37.956681 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:37.956656    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9cxdf"]
Jul 08 03:24:37.959179 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:37.959150    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3d498c1_68f2_4915_8504_80e20964154d.slice/crio-ffe8ebb0d09d4d020ca47a61cf5ad2812649314778b7cd9ac03a7760e6e4c31c WatchSource:0}: Error finding container ffe8ebb0d09d4d020ca47a61cf5ad2812649314778b7cd9ac03a7760e6e4c31c: Status 404 returned error can't find the container with id ffe8ebb0d09d4d020ca47a61cf5ad2812649314778b7cd9ac03a7760e6e4c31c
Jul 08 03:24:38.349576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:38.349539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfqgx" event={"ID":"047a2caa-fd23-46fb-9fbe-91cadb1a5788","Type":"ContainerStarted","Data":"f64e07cd040cb610afe14c884ddd797aec380c548aee01baa472cec1737465e4"}
Jul 08 03:24:38.350832 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:38.350807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9cxdf" event={"ID":"a3d498c1-68f2-4915-8504-80e20964154d","Type":"ContainerStarted","Data":"ffe8ebb0d09d4d020ca47a61cf5ad2812649314778b7cd9ac03a7760e6e4c31c"}
Jul 08 03:24:38.379583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:38.379554    2563 scope.go:117] "RemoveContainer" containerID="4d38f4a31ba17af16523d345bba6cb1d3da3a86d705824c3613f54c2915257eb"
Jul 08 03:24:39.358949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.358917    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:24:39.359439 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.359024    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" event={"ID":"788ca259-379f-4dfb-9f16-9fc1f443ecd8","Type":"ContainerStarted","Data":"5f3d166139cfb904593cd70f50a16cfc2d9447877cdc95d4d5e2cc9e971fd75d"}
Jul 08 03:24:39.359503 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.359491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:39.367033 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.366974    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-9lstn"
Jul 08 03:24:39.367322 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.367297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfqgx" event={"ID":"047a2caa-fd23-46fb-9fbe-91cadb1a5788","Type":"ContainerStarted","Data":"2fb88fab46bcced0d677a84a5460022736c75a04e9c8cdeee67e7d922f3b27f0"}
Jul 08 03:24:39.394023 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:39.393835    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-9lstn" podStartSLOduration=23.175620133 podStartE2EDuration="26.393813536s" totalImagesPullingTime="3.218193403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:39.393377338 +0000 UTC m=+290.501723279" watchObservedRunningTime="2026-07-08 03:24:39.393813536 +0000 UTC m=+290.502159481"
Jul 08 03:24:40.372840 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:40.372782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9cxdf" event={"ID":"a3d498c1-68f2-4915-8504-80e20964154d","Type":"ContainerStarted","Data":"8b0a08cc4289716b0f80a3cece84c6f8af3e6fdfc2fb7ac1fea704eb5c1e9740"}
Jul 08 03:24:40.373322 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:40.372904    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9cxdf" event={"ID":"a3d498c1-68f2-4915-8504-80e20964154d","Type":"ContainerStarted","Data":"6fd8d746be77b81e9922287a87f285b89dde7ce3d053c5e8398d495bd7e4e704"}
Jul 08 03:24:40.373322 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:40.373130    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:40.393334 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:40.393284    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9cxdf" podStartSLOduration=252.700204077 podStartE2EDuration="4m14.393263898s" totalImagesPullingTime="1.693059821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:20:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:40.392158723 +0000 UTC m=+291.500504665" watchObservedRunningTime="2026-07-08 03:24:40.393263898 +0000 UTC m=+291.501609823"
Jul 08 03:24:41.382690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:41.382632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mfqgx" event={"ID":"047a2caa-fd23-46fb-9fbe-91cadb1a5788","Type":"ContainerStarted","Data":"f8b907c57fc052890c213385dba564c3d506b3f7372efe595c6d7cf5b293f381"}
Jul 08 03:24:41.398542 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:41.398500    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mfqgx" podStartSLOduration=16.593427456 podStartE2EDuration="23.398483822s" totalImagesPullingTime="6.805056366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 03:24:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:41.397297181 +0000 UTC m=+292.505643122" watchObservedRunningTime="2026-07-08 03:24:41.398483822 +0000 UTC m=+292.506829793"
Jul 08 03:24:44.380544 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:44.380495    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-v5fdt" podUID="848ea393-51f5-40f4-919f-7b147f20ff22"
Jul 08 03:24:44.517700 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:44.517667    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:24:44.522401 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:44.522359    2563 patch_prober.go:28] interesting pod/image-registry-67499fc746-fm5gv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 03:24:44.522540 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:44.522421    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 03:24:45.242356 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:45.242323    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:45.242904 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:45.242876    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aff7f08a-06ac-41bb-9e76-14f8670b4c84-service-ca-bundle\") pod \"router-default-cf9c66499-8vtcz\" (UID: \"aff7f08a-06ac-41bb-9e76-14f8670b4c84\") " pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:45.431779 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:45.431742    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:45.579517 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:45.579487    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-cf9c66499-8vtcz"]
Jul 08 03:24:45.582332 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:45.582290    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaff7f08a_06ac_41bb_9e76_14f8670b4c84.slice/crio-21f2bea00d3b8390cfd3c62a217512e2a6d3dd94fad9bc09bccff9581eca8a83 WatchSource:0}: Error finding container 21f2bea00d3b8390cfd3c62a217512e2a6d3dd94fad9bc09bccff9581eca8a83: Status 404 returned error can't find the container with id 21f2bea00d3b8390cfd3c62a217512e2a6d3dd94fad9bc09bccff9581eca8a83
Jul 08 03:24:46.395115 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.395080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cf9c66499-8vtcz" event={"ID":"aff7f08a-06ac-41bb-9e76-14f8670b4c84","Type":"ContainerStarted","Data":"9a4d026bfecd653e27f9cf5c71763073ef696c1043cafdd818d21a074ffa4a52"}
Jul 08 03:24:46.395311 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.395161    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-cf9c66499-8vtcz" event={"ID":"aff7f08a-06ac-41bb-9e76-14f8670b4c84","Type":"ContainerStarted","Data":"21f2bea00d3b8390cfd3c62a217512e2a6d3dd94fad9bc09bccff9581eca8a83"}
Jul 08 03:24:46.416050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.415997    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-cf9c66499-8vtcz" podStartSLOduration=33.415980919 podStartE2EDuration="33.415980919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:46.41442104 +0000 UTC m=+297.522766979" watchObservedRunningTime="2026-07-08 03:24:46.415980919 +0000 UTC m=+297.524326859"
Jul 08 03:24:46.432167 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.432122    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:46.434568 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.434549    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:46.531730 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.531697    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:24:46.534886 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.534870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.537407 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537311    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 08 03:24:46.537407 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 08 03:24:46.537407 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537394    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 08 03:24:46.537407 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537403    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 08 03:24:46.537874 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537651    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 08 03:24:46.537874 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537690    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 08 03:24:46.538048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537971    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 08 03:24:46.538048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.537992    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j2b7c\""
Jul 08 03:24:46.544332 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.544313    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:24:46.654465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654373    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.654465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.654684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l44hd\" (UniqueName: \"kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.654684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654586    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.654684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.654684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.654645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755615 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l44hd\" (UniqueName: \"kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755615 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755696    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.755883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.755712    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.756571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.756505    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.756694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.756570    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.756694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.756593    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.758162 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.758109    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.758269 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.758254    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.764065 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.764046    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l44hd\" (UniqueName: \"kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd\") pod \"console-f86fc6744-cbm4j\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") " pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.844111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.844066    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:46.973837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:46.973806    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:24:46.977731 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:46.977705    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d5e2292_c198_43a9_ad4a_339eb4e3ff19.slice/crio-4c347b7328dd5ab468ac0bda550ab1a79f7c706d509ce710554cf2de7093c5ae WatchSource:0}: Error finding container 4c347b7328dd5ab468ac0bda550ab1a79f7c706d509ce710554cf2de7093c5ae: Status 404 returned error can't find the container with id 4c347b7328dd5ab468ac0bda550ab1a79f7c706d509ce710554cf2de7093c5ae
Jul 08 03:24:47.398877 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:47.398832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86fc6744-cbm4j" event={"ID":"6d5e2292-c198-43a9-ad4a-339eb4e3ff19","Type":"ContainerStarted","Data":"4c347b7328dd5ab468ac0bda550ab1a79f7c706d509ce710554cf2de7093c5ae"}
Jul 08 03:24:47.399024 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:47.399008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:47.400177 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:47.400160    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-cf9c66499-8vtcz"
Jul 08 03:24:49.889250 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:49.889220    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:24:49.889250 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:49.889235    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:24:49.893652 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:49.893634    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:24:49.893781 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:49.893757    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:24:49.900211 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:49.900194    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 03:24:50.380928 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:50.380896    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9cxdf"
Jul 08 03:24:50.411659 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:50.411624    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86fc6744-cbm4j" event={"ID":"6d5e2292-c198-43a9-ad4a-339eb4e3ff19","Type":"ContainerStarted","Data":"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"}
Jul 08 03:24:50.429026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:50.428980    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f86fc6744-cbm4j" podStartSLOduration=1.464322614 podStartE2EDuration="4.428964427s" totalImagesPullingTime="2.964641813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:24:50.427483366 +0000 UTC m=+301.535829303" watchObservedRunningTime="2026-07-08 03:24:50.428964427 +0000 UTC m=+301.537310443"
Jul 08 03:24:54.522816 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:54.522785    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:24:56.844583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:56.844540    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:56.844981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:56.844617    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:56.849403 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:56.849376    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:57.349967 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.349927    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-wl8vm"]
Jul 08 03:24:57.352878 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.352859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.356503 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356467    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 03:24:57.356713 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356593    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 03:24:57.356713 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356638    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 03:24:57.356899 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-msnx5\""
Jul 08 03:24:57.356986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 03:24:57.357055 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.356983    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 03:24:57.357180 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.357131    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 03:24:57.371756 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.371730    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"]
Jul 08 03:24:57.373848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.373834    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.376065 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.376035    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 08 03:24:57.376214 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.376047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-cn78f\""
Jul 08 03:24:57.376289 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.376047    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 08 03:24:57.376596 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.376577    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 08 03:24:57.391024 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.390973    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"]
Jul 08 03:24:57.439936 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.439907    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:24:57.450438 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450405    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-wtmp\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450438 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-root\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-metrics-client-ca\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450497    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-textfile\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450544    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tslzc\" (UniqueName: \"kubernetes.io/projected/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-kube-api-access-tslzc\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-sys\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.450675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.450671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-textfile\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552325 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552097    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.552325 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552433 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:57.552336    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 08 03:24:57.552433 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:57.552420    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls podName:f2d3735b-eea5-4dd0-a6b0-13cc4e84894d nodeName:}" failed. No retries permitted until 2026-07-08 03:24:58.052399412 +0000 UTC m=+309.160745333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls") pod "node-exporter-wl8vm" (UID: "f2d3735b-eea5-4dd0-a6b0-13cc4e84894d") : secret "node-exporter-tls" not found
Jul 08 03:24:57.552537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552476    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-textfile\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr9g2\" (UniqueName: \"kubernetes.io/projected/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-api-access-jr9g2\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.552667 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tslzc\" (UniqueName: \"kubernetes.io/projected/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-kube-api-access-tslzc\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552732 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-sys\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552803 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552784    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-sys\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552867 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.552994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552966    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.552994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.552981    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-accelerators-collector-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.553187 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553057    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.553187 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-wtmp\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.553410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553393    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.553484 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553430    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-wtmp\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.553598 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-root\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.553783 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553695    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-root\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.553898 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.553898 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.553841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-metrics-client-ca\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.554432 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.554412    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-metrics-client-ca\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.555610 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.555582    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.560899 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.560872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tslzc\" (UniqueName: \"kubernetes.io/projected/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-kube-api-access-tslzc\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:57.654993 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.654895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.654993 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.654942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr9g2\" (UniqueName: \"kubernetes.io/projected/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-api-access-jr9g2\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.654993 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.654982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.655298 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.655008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.655298 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.655041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.655298 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.655073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.655451 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:57.655331    2563 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 08 03:24:57.655503 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:24:57.655477    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls podName:0d972215-36e6-46db-a1ae-5bd18cb5b4aa nodeName:}" failed. No retries permitted until 2026-07-08 03:24:58.155394518 +0000 UTC m=+309.263740454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-ndjwj" (UID: "0d972215-36e6-46db-a1ae-5bd18cb5b4aa") : secret "kube-state-metrics-tls" not found
Jul 08 03:24:57.655630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.655603    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.656043 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.656023    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.656163 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.656119    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.657725 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.657697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:57.669446 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:57.669416    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr9g2\" (UniqueName: \"kubernetes.io/projected/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-api-access-jr9g2\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:58.057882 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.057846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:58.060260 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.060235    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f2d3735b-eea5-4dd0-a6b0-13cc4e84894d-node-exporter-tls\") pod \"node-exporter-wl8vm\" (UID: \"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d\") " pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:58.158651 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.158608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:58.161259 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.161232    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0d972215-36e6-46db-a1ae-5bd18cb5b4aa-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-ndjwj\" (UID: \"0d972215-36e6-46db-a1ae-5bd18cb5b4aa\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:58.263320 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.263279    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-wl8vm"
Jul 08 03:24:58.273427 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:58.273395    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d3735b_eea5_4dd0_a6b0_13cc4e84894d.slice/crio-d307abf06d28fe6eee2e0f3cd0e5eb8f3442e61581cad2fe9d8d6c28d4b5a53f WatchSource:0}: Error finding container d307abf06d28fe6eee2e0f3cd0e5eb8f3442e61581cad2fe9d8d6c28d4b5a53f: Status 404 returned error can't find the container with id d307abf06d28fe6eee2e0f3cd0e5eb8f3442e61581cad2fe9d8d6c28d4b5a53f
Jul 08 03:24:58.275084 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.275068    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:24:58.283745 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.283724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"
Jul 08 03:24:58.379771 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.378733    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:24:58.432524 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.432488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj"]
Jul 08 03:24:58.435842 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:58.435813    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d972215_36e6_46db_a1ae_5bd18cb5b4aa.slice/crio-3b03249cf85b7acea63d8fb41bb6c030ca919ef8f2f5091950f743382e621f88 WatchSource:0}: Error finding container 3b03249cf85b7acea63d8fb41bb6c030ca919ef8f2f5091950f743382e621f88: Status 404 returned error can't find the container with id 3b03249cf85b7acea63d8fb41bb6c030ca919ef8f2f5091950f743382e621f88
Jul 08 03:24:58.439842 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.439807    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl8vm" event={"ID":"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d","Type":"ContainerStarted","Data":"d307abf06d28fe6eee2e0f3cd0e5eb8f3442e61581cad2fe9d8d6c28d4b5a53f"}
Jul 08 03:24:58.497462 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.497420    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 03:24:58.500916 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.500897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.503545 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.503520    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 08 03:24:58.503654 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.503585    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 08 03:24:58.504544 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504527    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 08 03:24:58.504781 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504761    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 08 03:24:58.504896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504761    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 08 03:24:58.504896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504856    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mzk54\""
Jul 08 03:24:58.504896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504881    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 08 03:24:58.505066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504944    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 08 03:24:58.505066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.504957    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 08 03:24:58.505201 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.505075    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 08 03:24:58.520731 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.520706    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 03:24:58.664630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664556    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-config-out\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664737    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664916    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.664949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664938    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.665061 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664962    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tblbs\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-kube-api-access-tblbs\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.665061 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.664985    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-web-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766025 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.765991    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-config-out\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766104    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766515 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766515 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766312    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766515 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766515 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.766515 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.766410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.767340 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.767010    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.767442 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.767358    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tblbs\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-kube-api-access-tblbs\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.767442 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.767404    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-web-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.767557 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.767443    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.767759 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.767733    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.769712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.769683    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.769877 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.769854    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.770197 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.770171    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/421626d6-5819-4a78-9555-8dbff83165c9-config-out\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.770287 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.770268    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-tls-assets\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.770347 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.770281    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-config-volume\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.771019 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.770997    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.771304 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.771282    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.772338 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.772318    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-web-config\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.772522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.772503    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/421626d6-5819-4a78-9555-8dbff83165c9-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.779910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.779883    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tblbs\" (UniqueName: \"kubernetes.io/projected/421626d6-5819-4a78-9555-8dbff83165c9-kube-api-access-tblbs\") pod \"alertmanager-main-0\" (UID: \"421626d6-5819-4a78-9555-8dbff83165c9\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.810784 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.810753    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 08 03:24:58.971916 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:58.971890    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 08 03:24:58.974521 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:24:58.974491    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421626d6_5819_4a78_9555_8dbff83165c9.slice/crio-b879405b44be5825a22ec4e24db672bd0e64bd98ed54c8ac53f6fa2acd2e7d40 WatchSource:0}: Error finding container b879405b44be5825a22ec4e24db672bd0e64bd98ed54c8ac53f6fa2acd2e7d40: Status 404 returned error can't find the container with id b879405b44be5825a22ec4e24db672bd0e64bd98ed54c8ac53f6fa2acd2e7d40
Jul 08 03:24:59.445414 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:59.445336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj" event={"ID":"0d972215-36e6-46db-a1ae-5bd18cb5b4aa","Type":"ContainerStarted","Data":"3b03249cf85b7acea63d8fb41bb6c030ca919ef8f2f5091950f743382e621f88"}
Jul 08 03:24:59.447283 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:59.447204    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2d3735b-eea5-4dd0-a6b0-13cc4e84894d" containerID="35e48852fa8e6659acd20347db274c2883d5db5e36ab6143d28a2f7b047cc24f" exitCode=0
Jul 08 03:24:59.447404 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:59.447296    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl8vm" event={"ID":"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d","Type":"ContainerDied","Data":"35e48852fa8e6659acd20347db274c2883d5db5e36ab6143d28a2f7b047cc24f"}
Jul 08 03:24:59.449281 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:24:59.449257    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"b879405b44be5825a22ec4e24db672bd0e64bd98ed54c8ac53f6fa2acd2e7d40"}
Jul 08 03:25:00.459296 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.459251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl8vm" event={"ID":"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d","Type":"ContainerStarted","Data":"562f917db6b476feb1ab4f475b86f904d7ad5985599a3345424f22115df55661"}
Jul 08 03:25:00.459740 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.459358    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-wl8vm" event={"ID":"f2d3735b-eea5-4dd0-a6b0-13cc4e84894d","Type":"ContainerStarted","Data":"8148e7b280bf1d71da83c613d5339453bd83f372a3688dd9c88c79772153176a"}
Jul 08 03:25:00.460809 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.460783    2563 generic.go:356] "Generic (PLEG): container finished" podID="421626d6-5819-4a78-9555-8dbff83165c9" containerID="22aac9a96315c828549832ac7bde6a8a36a6efd29e77d3f12bff24055e724bd0" exitCode=0
Jul 08 03:25:00.460922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.460863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerDied","Data":"22aac9a96315c828549832ac7bde6a8a36a6efd29e77d3f12bff24055e724bd0"}
Jul 08 03:25:00.463198 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.463174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj" event={"ID":"0d972215-36e6-46db-a1ae-5bd18cb5b4aa","Type":"ContainerStarted","Data":"93ecf70dc162df951f885aaa4491bc745b29742550f040ea4c82807a47dacbcc"}
Jul 08 03:25:00.463282 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.463220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj" event={"ID":"0d972215-36e6-46db-a1ae-5bd18cb5b4aa","Type":"ContainerStarted","Data":"3bf4f72df25fbe50417b7b5203808d7e0030714c9d2f111896a3a96394ba2831"}
Jul 08 03:25:00.463282 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.463243    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj" event={"ID":"0d972215-36e6-46db-a1ae-5bd18cb5b4aa","Type":"ContainerStarted","Data":"460659cc55c8537c0a07e606449ae76704f7c05b16fe2353b192828b8a80908f"}
Jul 08 03:25:00.486918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.486839    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-wl8vm" podStartSLOduration=2.445027225 podStartE2EDuration="3.486820229s" totalImagesPullingTime="958.646711ms" totalInitContainerRuntime="83.146293ms" isStatefulPod=true podCreationTimestamp="2026-07-08 03:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:00.486682931 +0000 UTC m=+311.595028871" watchObservedRunningTime="2026-07-08 03:25:00.486820229 +0000 UTC m=+311.595166170"
Jul 08 03:25:00.511309 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:00.511248    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-ndjwj" podStartSLOduration=2.129175481 podStartE2EDuration="3.511227784s" totalImagesPullingTime="1.382052303s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:00.509750652 +0000 UTC m=+311.618096593" watchObservedRunningTime="2026-07-08 03:25:00.511227784 +0000 UTC m=+311.619573725"
Jul 08 03:25:03.478948 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:03.478912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"e6283b562eafb639d431a6ce6ad79744d41415d45c0c97c81a237b77d12fce31"}
Jul 08 03:25:03.479379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:03.479009    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"3d9585c00bb096302f7524e0827058de0bb1f802875d5444ba3fc2acb607f939"}
Jul 08 03:25:03.479379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:03.479045    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"a9ff5f448b01c0a0330f1bf16cd1177d677a023f391bdbc26ea26bbce9e28f72"}
Jul 08 03:25:03.479379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:03.479069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"c901e28434f91a33fbc0ca5657c03a64b57ee51c319f36ceb3f313a2900bc5a0"}
Jul 08 03:25:03.479379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:03.479091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"929f5696b4f32a474839327535b7509b3f10d7bc29b9ee147b359d1e0f1d7976"}
Jul 08 03:25:04.486200 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:04.486165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"421626d6-5819-4a78-9555-8dbff83165c9","Type":"ContainerStarted","Data":"ecbc80f84a955241871508a378d750684177bbe9ccd46d26bd857d03b394c7e6"}
Jul 08 03:25:04.524869 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:04.524810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.178527919 podStartE2EDuration="6.524789756s" totalImagesPullingTime="4.33006448s" totalInitContainerRuntime="16.197357ms" isStatefulPod=false podCreationTimestamp="2026-07-08 03:24:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:04.522822098 +0000 UTC m=+315.631168085" watchObservedRunningTime="2026-07-08 03:25:04.524789756 +0000 UTC m=+315.633135696"
Jul 08 03:25:09.538183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.538110    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerName="registry" containerID="cri-o://5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff" gracePeriod=30
Jul 08 03:25:09.787721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.787697    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:25:09.872852 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872816    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872876    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872901    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872973    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.872993    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.873015    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873048 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.873043    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rpgvg\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg\") pod \"5f564f08-1db0-496d-b77d-2a7d1cc98203\" (UID: \"5f564f08-1db0-496d-b77d-2a7d1cc98203\") "
Jul 08 03:25:09.873504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.873446    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:09.873504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.873467    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:09.875829 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.875780    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:09.875926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.875907    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg" (OuterVolumeSpecName: "kube-api-access-rpgvg") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "kube-api-access-rpgvg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:09.875926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.875909    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:09.876038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.875924    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:09.876038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.875896    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:09.881826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.881801    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "5f564f08-1db0-496d-b77d-2a7d1cc98203" (UID: "5f564f08-1db0-496d-b77d-2a7d1cc98203"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:25:09.974566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974511    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-image-registry-private-configuration\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974556    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-trusted-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974568    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/5f564f08-1db0-496d-b77d-2a7d1cc98203-ca-trust-extracted\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974577    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-certificates\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974587    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-registry-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974596    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-bound-sa-token\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974603    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpgvg\" (UniqueName: \"kubernetes.io/projected/5f564f08-1db0-496d-b77d-2a7d1cc98203-kube-api-access-rpgvg\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:09.974848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:09.974611    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/5f564f08-1db0-496d-b77d-2a7d1cc98203-installation-pull-secrets\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:10.509962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.509926    2563 generic.go:356] "Generic (PLEG): container finished" podID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerID="5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff" exitCode=0
Jul 08 03:25:10.509962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.509968    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67499fc746-fm5gv"
Jul 08 03:25:10.510321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.509973    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" event={"ID":"5f564f08-1db0-496d-b77d-2a7d1cc98203","Type":"ContainerDied","Data":"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"}
Jul 08 03:25:10.510321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.510020    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67499fc746-fm5gv" event={"ID":"5f564f08-1db0-496d-b77d-2a7d1cc98203","Type":"ContainerDied","Data":"d280b8ef2c13361f17b1dfd60de08cb4dac00372dbf5ff40010446bbe61553f6"}
Jul 08 03:25:10.510321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.510040    2563 scope.go:117] "RemoveContainer" containerID="5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"
Jul 08 03:25:10.518387 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.518373    2563 scope.go:117] "RemoveContainer" containerID="5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"
Jul 08 03:25:10.518629 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:25:10.518610    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff\": container with ID starting with 5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff not found: ID does not exist" containerID="5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"
Jul 08 03:25:10.518678 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.518638    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff"} err="failed to get container status \"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff\": rpc error: code = NotFound desc = could not find container \"5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff\": container with ID starting with 5cad986f1dc9d8dade4da86a5eba55eafe95fbb643f7ff7f49e9af51b763baff not found: ID does not exist"
Jul 08 03:25:10.531946 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.531920    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:25:10.535851 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:10.535830    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67499fc746-fm5gv"]
Jul 08 03:25:11.383380 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:11.383351    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" path="/var/lib/kubelet/pods/5f564f08-1db0-496d-b77d-2a7d1cc98203/volumes"
Jul 08 03:25:13.322091 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.322050    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:25:13.322589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.322465    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerName="registry"
Jul 08 03:25:13.322589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.322482    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerName="registry"
Jul 08 03:25:13.322589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.322565    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f564f08-1db0-496d-b77d-2a7d1cc98203" containerName="registry"
Jul 08 03:25:13.327508 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.327487    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.340327 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.337464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:25:13.340327 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.338496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 08 03:25:13.400915 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.400872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.400925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.401004    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdnml\" (UniqueName: \"kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.401048    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.401071    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401286 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.401171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.401286 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.401202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502606 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502606 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502639    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502725    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdnml\" (UniqueName: \"kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.502889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.502816    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.503583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.503556    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.503696 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.503602    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.503987 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.503962    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.504026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.503990    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.505420 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.505399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.505489 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.505457    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.511633 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.511607    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdnml\" (UniqueName: \"kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml\") pod \"console-6f89c68cc5-nbxsh\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") " pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.638376 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.638281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:13.773503 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:13.773478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:25:13.775837 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:25:13.775810    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07559bb1_13d7_470e_998b_ec3c49214208.slice/crio-fd65af915e46bc2e13d72a07e1d1ca6943435a2baa7b676e5b0e00cfb29892a5 WatchSource:0}: Error finding container fd65af915e46bc2e13d72a07e1d1ca6943435a2baa7b676e5b0e00cfb29892a5: Status 404 returned error can't find the container with id fd65af915e46bc2e13d72a07e1d1ca6943435a2baa7b676e5b0e00cfb29892a5
Jul 08 03:25:14.527133 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:14.527099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f89c68cc5-nbxsh" event={"ID":"07559bb1-13d7-470e-998b-ec3c49214208","Type":"ContainerStarted","Data":"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"}
Jul 08 03:25:14.527569 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:14.527169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f89c68cc5-nbxsh" event={"ID":"07559bb1-13d7-470e-998b-ec3c49214208","Type":"ContainerStarted","Data":"fd65af915e46bc2e13d72a07e1d1ca6943435a2baa7b676e5b0e00cfb29892a5"}
Jul 08 03:25:14.548551 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:14.548500    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6f89c68cc5-nbxsh" podStartSLOduration=1.548483338 podStartE2EDuration="1.548483338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:25:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:25:14.54755654 +0000 UTC m=+325.655902491" watchObservedRunningTime="2026-07-08 03:25:14.548483338 +0000 UTC m=+325.656829277"
Jul 08 03:25:23.565099 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.565064    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f8cdf2b-e0ba-4223-aa42-431ec6ac0814" containerID="2e57a4c95ba3202048a078e9a2ab186ddd0a57944248398e03c9942c1034c5f4" exitCode=0
Jul 08 03:25:23.565539 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.565154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp" event={"ID":"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814","Type":"ContainerDied","Data":"2e57a4c95ba3202048a078e9a2ab186ddd0a57944248398e03c9942c1034c5f4"}
Jul 08 03:25:23.565539 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.565522    2563 scope.go:117] "RemoveContainer" containerID="2e57a4c95ba3202048a078e9a2ab186ddd0a57944248398e03c9942c1034c5f4"
Jul 08 03:25:23.566769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.566734    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf02e941-4d09-4510-86f4-7035ff6ff81c" containerID="5dfdcc0e24343905c02b18d0ce0ba5b9473b3dee94381bbdb0fd3bb73dedf180" exitCode=0
Jul 08 03:25:23.566903 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.566801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h" event={"ID":"cf02e941-4d09-4510-86f4-7035ff6ff81c","Type":"ContainerDied","Data":"5dfdcc0e24343905c02b18d0ce0ba5b9473b3dee94381bbdb0fd3bb73dedf180"}
Jul 08 03:25:23.567122 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.567106    2563 scope.go:117] "RemoveContainer" containerID="5dfdcc0e24343905c02b18d0ce0ba5b9473b3dee94381bbdb0fd3bb73dedf180"
Jul 08 03:25:23.639426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.639399    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:23.639648 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.639630    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:23.644695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:23.644673    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:24.246587 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.246560    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cf9c66499-8vtcz_aff7f08a-06ac-41bb-9e76-14f8670b4c84/router/0.log"
Jul 08 03:25:24.259276 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.259253    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mbkbh_d19979df-55c8-4e1e-8a25-0390633d1794/serve-healthcheck-canary/0.log"
Jul 08 03:25:24.572372 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.572335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-r6rvp" event={"ID":"2f8cdf2b-e0ba-4223-aa42-431ec6ac0814","Type":"ContainerStarted","Data":"2c4b6095a81abd36076057626a538ebd486bcc4e311d51161510c0f82e2fc1ee"}
Jul 08 03:25:24.574013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.573991    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-6cs8h" event={"ID":"cf02e941-4d09-4510-86f4-7035ff6ff81c","Type":"ContainerStarted","Data":"707361b7921019e9fd78c352f74d9614522060d1d9603e2d945825e0787faa14"}
Jul 08 03:25:24.577820 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.577800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:25:24.633952 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:24.633923    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:25:49.656779 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:49.656725    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-f86fc6744-cbm4j" podUID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" containerName="console" containerID="cri-o://5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6" gracePeriod=15
Jul 08 03:25:49.899176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:49.899131    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:25:50.024952 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.024862    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.024952 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.024919    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l44hd\" (UniqueName: \"kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.025201 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.024963    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.025201 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.024987    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.025201 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.025008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.025201 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.025027    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert\") pod \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\" (UID: \"6d5e2292-c198-43a9-ad4a-339eb4e3ff19\") "
Jul 08 03:25:50.025457 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.025368    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca" (OuterVolumeSpecName: "service-ca") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:50.025543 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.025515    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:50.025543 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.025527    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config" (OuterVolumeSpecName: "console-config") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:25:50.027527 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.027503    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:50.027527 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.027518    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd" (OuterVolumeSpecName: "kube-api-access-l44hd") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "kube-api-access-l44hd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:25:50.027670 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.027562    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6d5e2292-c198-43a9-ad4a-339eb4e3ff19" (UID: "6d5e2292-c198-43a9-ad4a-339eb4e3ff19"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:25:50.125597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125565    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l44hd\" (UniqueName: \"kubernetes.io/projected/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-kube-api-access-l44hd\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.125597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125595    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.125799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125608    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-oauth-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.125799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125620    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-oauth-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.125799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125631    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-console-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.125799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.125643    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d5e2292-c198-43a9-ad4a-339eb4e3ff19-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:25:50.674203 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.674168    2563 generic.go:356] "Generic (PLEG): container finished" podID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" containerID="5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6" exitCode=0
Jul 08 03:25:50.674203 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.674207    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f86fc6744-cbm4j"
Jul 08 03:25:50.674757 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.674207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86fc6744-cbm4j" event={"ID":"6d5e2292-c198-43a9-ad4a-339eb4e3ff19","Type":"ContainerDied","Data":"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"}
Jul 08 03:25:50.674757 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.674338    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f86fc6744-cbm4j" event={"ID":"6d5e2292-c198-43a9-ad4a-339eb4e3ff19","Type":"ContainerDied","Data":"4c347b7328dd5ab468ac0bda550ab1a79f7c706d509ce710554cf2de7093c5ae"}
Jul 08 03:25:50.674757 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.674364    2563 scope.go:117] "RemoveContainer" containerID="5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"
Jul 08 03:25:50.685293 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.685243    2563 scope.go:117] "RemoveContainer" containerID="5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"
Jul 08 03:25:50.685660 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:25:50.685640    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6\": container with ID starting with 5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6 not found: ID does not exist" containerID="5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"
Jul 08 03:25:50.685714 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.685669    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6"} err="failed to get container status \"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6\": rpc error: code = NotFound desc = could not find container \"5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6\": container with ID starting with 5a7a6fed6deccf367935cebbf7aacbc57a3f0aa64db8972ebc2aa991967bf6c6 not found: ID does not exist"
Jul 08 03:25:50.695347 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.695329    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:25:50.701003 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:50.700985    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f86fc6744-cbm4j"]
Jul 08 03:25:51.383325 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:25:51.383295    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" path="/var/lib/kubelet/pods/6d5e2292-c198-43a9-ad4a-339eb4e3ff19/volumes"
Jul 08 03:26:03.348364 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.348324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:26:03.350758 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.350739    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/848ea393-51f5-40f4-919f-7b147f20ff22-metrics-certs\") pod \"network-metrics-daemon-v5fdt\" (UID: \"848ea393-51f5-40f4-919f-7b147f20ff22\") " pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:26:03.482291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.482260    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zb8ld\""
Jul 08 03:26:03.490650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.490629    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v5fdt"
Jul 08 03:26:03.619610 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.619488    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v5fdt"]
Jul 08 03:26:03.621878 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:26:03.621844    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod848ea393_51f5_40f4_919f_7b147f20ff22.slice/crio-1ba20eeac259097dcab12063c7b552886da178b53ef6cc3b10a2cd6ce4bf10c0 WatchSource:0}: Error finding container 1ba20eeac259097dcab12063c7b552886da178b53ef6cc3b10a2cd6ce4bf10c0: Status 404 returned error can't find the container with id 1ba20eeac259097dcab12063c7b552886da178b53ef6cc3b10a2cd6ce4bf10c0
Jul 08 03:26:03.721949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:03.721919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5fdt" event={"ID":"848ea393-51f5-40f4-919f-7b147f20ff22","Type":"ContainerStarted","Data":"1ba20eeac259097dcab12063c7b552886da178b53ef6cc3b10a2cd6ce4bf10c0"}
Jul 08 03:26:04.727868 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:04.727781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5fdt" event={"ID":"848ea393-51f5-40f4-919f-7b147f20ff22","Type":"ContainerStarted","Data":"c8427449053fb2239579da86a8474d96cbc6b18742580646899cf80e98719953"}
Jul 08 03:26:04.727868 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:04.727831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v5fdt" event={"ID":"848ea393-51f5-40f4-919f-7b147f20ff22","Type":"ContainerStarted","Data":"4c6ab811d1e1ac925e4a7356c3918968bd7e7a381acb8803cdce7411318ac3ba"}
Jul 08 03:26:04.745887 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:04.745837    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v5fdt" podStartSLOduration=373.894032881 podStartE2EDuration="6m14.745821235s" totalImagesPullingTime="851.788354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:26:04.744046195 +0000 UTC m=+375.852392135" watchObservedRunningTime="2026-07-08 03:26:04.745821235 +0000 UTC m=+375.854167173"
Jul 08 03:26:23.701045 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.700959    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:26:23.701522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.701267    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" containerName="console"
Jul 08 03:26:23.701522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.701279    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" containerName="console"
Jul 08 03:26:23.701522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.701337    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d5e2292-c198-43a9-ad4a-339eb4e3ff19" containerName="console"
Jul 08 03:26:23.705111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.705093    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718355    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718387    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gz7v\" (UniqueName: \"kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718407    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718632 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.718632 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.718457    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.729946 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.729920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:26:23.819227 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gz7v\" (UniqueName: \"kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819227 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819229    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819287    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.819472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.819405    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.820106 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.820078    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.820259 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.820179    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.820308 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.820270    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.820406 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.820382    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.821876 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.821859    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.821939 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.821920    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:23.833011 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:23.832989    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gz7v\" (UniqueName: \"kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v\") pod \"console-56fb6bb6f5-vrh5k\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") " pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:24.014528 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:24.014441    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:24.148547 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:24.148089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:26:24.152965 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:26:24.152933    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode172fd0d_9d45_4d41_af3e_e91fc6ee996c.slice/crio-f373d8cf77c70e9c4e043559c359e780a0ed0ade4a1c765d46cdbc2f609bb258 WatchSource:0}: Error finding container f373d8cf77c70e9c4e043559c359e780a0ed0ade4a1c765d46cdbc2f609bb258: Status 404 returned error can't find the container with id f373d8cf77c70e9c4e043559c359e780a0ed0ade4a1c765d46cdbc2f609bb258
Jul 08 03:26:24.805011 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:24.804976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fb6bb6f5-vrh5k" event={"ID":"e172fd0d-9d45-4d41-af3e-e91fc6ee996c","Type":"ContainerStarted","Data":"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"}
Jul 08 03:26:24.805425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:24.805026    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fb6bb6f5-vrh5k" event={"ID":"e172fd0d-9d45-4d41-af3e-e91fc6ee996c","Type":"ContainerStarted","Data":"f373d8cf77c70e9c4e043559c359e780a0ed0ade4a1c765d46cdbc2f609bb258"}
Jul 08 03:26:24.822904 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:24.822842    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-56fb6bb6f5-vrh5k" podStartSLOduration=1.822827532 podStartE2EDuration="1.822827532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:26:24.82181875 +0000 UTC m=+395.930164729" watchObservedRunningTime="2026-07-08 03:26:24.822827532 +0000 UTC m=+395.931173473"
Jul 08 03:26:34.014826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:34.014787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:34.015258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:34.014855    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:34.019351 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:34.019330    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:34.851579 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:34.851543    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:26:34.898287 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:34.898254    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:26:59.926766 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:26:59.926726    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6f89c68cc5-nbxsh" podUID="07559bb1-13d7-470e-998b-ec3c49214208" containerName="console" containerID="cri-o://68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c" gracePeriod=15
Jul 08 03:27:00.166960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.166937    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:27:00.311739 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311718    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.311900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311752    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.311900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311777    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.311900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311813    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.311900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311864    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.312080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311913    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdnml\" (UniqueName: \"kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.312080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.311936    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle\") pod \"07559bb1-13d7-470e-998b-ec3c49214208\" (UID: \"07559bb1-13d7-470e-998b-ec3c49214208\") "
Jul 08 03:27:00.312264 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.312241    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config" (OuterVolumeSpecName: "console-config") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:27:00.312264 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.312178    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca" (OuterVolumeSpecName: "service-ca") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:27:00.312405 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.312380    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:27:00.312475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.312440    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:27:00.314170 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.314130    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:27:00.314249 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.314182    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:27:00.314308 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.314281    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml" (OuterVolumeSpecName: "kube-api-access-cdnml") pod "07559bb1-13d7-470e-998b-ec3c49214208" (UID: "07559bb1-13d7-470e-998b-ec3c49214208"). InnerVolumeSpecName "kube-api-access-cdnml". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:27:00.413589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413549    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413581    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-console-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413594    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413606    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/07559bb1-13d7-470e-998b-ec3c49214208-console-oauth-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413619    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-oauth-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413631    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdnml\" (UniqueName: \"kubernetes.io/projected/07559bb1-13d7-470e-998b-ec3c49214208-kube-api-access-cdnml\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.413821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.413641    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/07559bb1-13d7-470e-998b-ec3c49214208-trusted-ca-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:00.947806 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.947774    2563 generic.go:356] "Generic (PLEG): container finished" podID="07559bb1-13d7-470e-998b-ec3c49214208" containerID="68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c" exitCode=0
Jul 08 03:27:00.948258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.947821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f89c68cc5-nbxsh" event={"ID":"07559bb1-13d7-470e-998b-ec3c49214208","Type":"ContainerDied","Data":"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"}
Jul 08 03:27:00.948258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.947839    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6f89c68cc5-nbxsh"
Jul 08 03:27:00.948258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.947888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6f89c68cc5-nbxsh" event={"ID":"07559bb1-13d7-470e-998b-ec3c49214208","Type":"ContainerDied","Data":"fd65af915e46bc2e13d72a07e1d1ca6943435a2baa7b676e5b0e00cfb29892a5"}
Jul 08 03:27:00.948258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.947907    2563 scope.go:117] "RemoveContainer" containerID="68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"
Jul 08 03:27:00.956332 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.956314    2563 scope.go:117] "RemoveContainer" containerID="68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"
Jul 08 03:27:00.956597 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:27:00.956579    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c\": container with ID starting with 68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c not found: ID does not exist" containerID="68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"
Jul 08 03:27:00.956643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.956606    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c"} err="failed to get container status \"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c\": rpc error: code = NotFound desc = could not find container \"68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c\": container with ID starting with 68f236f12beda8621df2a7b84826a89970b6bd8893559254293b5504a056774c not found: ID does not exist"
Jul 08 03:27:00.969303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.969281    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:27:00.975614 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:00.975594    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6f89c68cc5-nbxsh"]
Jul 08 03:27:01.272312 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.272230    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"]
Jul 08 03:27:01.272518 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.272506    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07559bb1-13d7-470e-998b-ec3c49214208" containerName="console"
Jul 08 03:27:01.272576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.272519    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="07559bb1-13d7-470e-998b-ec3c49214208" containerName="console"
Jul 08 03:27:01.272576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.272563    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="07559bb1-13d7-470e-998b-ec3c49214208" containerName="console"
Jul 08 03:27:01.276712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.276696    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.279284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.279263    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7wtxf\""
Jul 08 03:27:01.279412 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.279267    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 08 03:27:01.279412 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.279295    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 08 03:27:01.285239 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.285179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"]
Jul 08 03:27:01.384425 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.384392    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07559bb1-13d7-470e-998b-ec3c49214208" path="/var/lib/kubelet/pods/07559bb1-13d7-470e-998b-ec3c49214208/volumes"
Jul 08 03:27:01.423848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.423810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v4w9\" (UniqueName: \"kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.424004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.423861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.424004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.423895    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.524982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.524887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v4w9\" (UniqueName: \"kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.524982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.524932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.525256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.525115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.525348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.525330    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.525423 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.525405    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.533587 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.533567    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v4w9\" (UniqueName: \"kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.589849 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.589817    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:01.717996 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.717972    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"]
Jul 08 03:27:01.720285 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:01.720262    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca73b54e_9ae3_40a9_8479_65892237daf7.slice/crio-229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98 WatchSource:0}: Error finding container 229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98: Status 404 returned error can't find the container with id 229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98
Jul 08 03:27:01.953886 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:01.953848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt" event={"ID":"ca73b54e-9ae3-40a9-8479-65892237daf7","Type":"ContainerStarted","Data":"229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98"}
Jul 08 03:27:07.982910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:07.982864    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerID="9a050d89b0044a019b83f864ef980f74f69ec8c29cf08c4d4adce51af49c12a4" exitCode=0
Jul 08 03:27:07.983593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:07.982919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt" event={"ID":"ca73b54e-9ae3-40a9-8479-65892237daf7","Type":"ContainerDied","Data":"9a050d89b0044a019b83f864ef980f74f69ec8c29cf08c4d4adce51af49c12a4"}
Jul 08 03:27:10.996918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:10.996880    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerID="74509b1a43ff862b837e4face5d07b61569e2db60c10dbb4b499744c7a1f7118" exitCode=0
Jul 08 03:27:10.997332 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:10.996952    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt" event={"ID":"ca73b54e-9ae3-40a9-8479-65892237daf7","Type":"ContainerDied","Data":"74509b1a43ff862b837e4face5d07b61569e2db60c10dbb4b499744c7a1f7118"}
Jul 08 03:27:19.033279 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:19.033241    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerID="f9bdd3a0e0b624621c5a8b761b40464cfa401b6e69cdc2f62122b89146a617d1" exitCode=0
Jul 08 03:27:19.033836 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:19.033315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt" event={"ID":"ca73b54e-9ae3-40a9-8479-65892237daf7","Type":"ContainerDied","Data":"f9bdd3a0e0b624621c5a8b761b40464cfa401b6e69cdc2f62122b89146a617d1"}
Jul 08 03:27:20.160806 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.160773    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:20.186923 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.186889    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle\") pod \"ca73b54e-9ae3-40a9-8479-65892237daf7\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") "
Jul 08 03:27:20.187080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.186948    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8v4w9\" (UniqueName: \"kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9\") pod \"ca73b54e-9ae3-40a9-8479-65892237daf7\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") "
Jul 08 03:27:20.187080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.186974    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util\") pod \"ca73b54e-9ae3-40a9-8479-65892237daf7\" (UID: \"ca73b54e-9ae3-40a9-8479-65892237daf7\") "
Jul 08 03:27:20.187721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.187689    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle" (OuterVolumeSpecName: "bundle") pod "ca73b54e-9ae3-40a9-8479-65892237daf7" (UID: "ca73b54e-9ae3-40a9-8479-65892237daf7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:27:20.189268 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.189236    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9" (OuterVolumeSpecName: "kube-api-access-8v4w9") pod "ca73b54e-9ae3-40a9-8479-65892237daf7" (UID: "ca73b54e-9ae3-40a9-8479-65892237daf7"). InnerVolumeSpecName "kube-api-access-8v4w9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:27:20.191235 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.191215    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util" (OuterVolumeSpecName: "util") pod "ca73b54e-9ae3-40a9-8479-65892237daf7" (UID: "ca73b54e-9ae3-40a9-8479-65892237daf7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:27:20.287676 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.287636    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:20.287676 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.287667    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8v4w9\" (UniqueName: \"kubernetes.io/projected/ca73b54e-9ae3-40a9-8479-65892237daf7-kube-api-access-8v4w9\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:20.287676 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:20.287677    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca73b54e-9ae3-40a9-8479-65892237daf7-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:21.043457 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:21.043428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt"
Jul 08 03:27:21.043620 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:21.043459    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408tfpt" event={"ID":"ca73b54e-9ae3-40a9-8479-65892237daf7","Type":"ContainerDied","Data":"229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98"}
Jul 08 03:27:21.043620 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:21.043503    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="229beb83aff31e29ec64e2f1851776852f352580af40d63bb0cbf325a3a7ef98"
Jul 08 03:27:24.262373 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262341    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"]
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262655    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="pull"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262667    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="pull"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262678    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="extract"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262687    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="extract"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262703    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="util"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262709    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="util"
Jul 08 03:27:24.262772 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.262764    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca73b54e-9ae3-40a9-8479-65892237daf7" containerName="extract"
Jul 08 03:27:24.266132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.266116    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.271522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.271501    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 08 03:27:24.271700 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.271681    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:27:24.271896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.271880    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-png8q\""
Jul 08 03:27:24.281038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.281014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"]
Jul 08 03:27:24.326625 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.326593    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.326625 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.326630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zgvm\" (UniqueName: \"kubernetes.io/projected/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-kube-api-access-5zgvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.427408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.427371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.427408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.427408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zgvm\" (UniqueName: \"kubernetes.io/projected/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-kube-api-access-5zgvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.427768 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.427746    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.438716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.438692    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zgvm\" (UniqueName: \"kubernetes.io/projected/c93fc34a-85d8-48fd-b8b2-f43888d3c9f1-kube-api-access-5zgvm\") pod \"cert-manager-operator-controller-manager-55c444bf86-zxnl5\" (UID: \"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.575117 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.575084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"
Jul 08 03:27:24.703693 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:24.703665    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5"]
Jul 08 03:27:24.707498 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:24.707474    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc93fc34a_85d8_48fd_b8b2_f43888d3c9f1.slice/crio-e9a7b53cde1c6036cf5c379d7f571e8d627fe04cca6e9e482d482417ac716fd6 WatchSource:0}: Error finding container e9a7b53cde1c6036cf5c379d7f571e8d627fe04cca6e9e482d482417ac716fd6: Status 404 returned error can't find the container with id e9a7b53cde1c6036cf5c379d7f571e8d627fe04cca6e9e482d482417ac716fd6
Jul 08 03:27:25.059909 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:25.059875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5" event={"ID":"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1","Type":"ContainerStarted","Data":"e9a7b53cde1c6036cf5c379d7f571e8d627fe04cca6e9e482d482417ac716fd6"}
Jul 08 03:27:27.072257 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:27.072218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5" event={"ID":"c93fc34a-85d8-48fd-b8b2-f43888d3c9f1","Type":"ContainerStarted","Data":"edd08b3c04272312bc2fa81e326937e762a722b834860d0ca37b31c2f3bbd148"}
Jul 08 03:27:27.096950 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:27.096895    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-zxnl5" podStartSLOduration=1.291847094 podStartE2EDuration="3.096878477s" totalImagesPullingTime="1.805031383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:27:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:27:27.096837946 +0000 UTC m=+458.205183886" watchObservedRunningTime="2026-07-08 03:27:27.096878477 +0000 UTC m=+458.205224399"
Jul 08 03:27:28.436223 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.436188    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rrf86"]
Jul 08 03:27:28.438499 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.438475    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.441178 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.441158    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bxmvl\""
Jul 08 03:27:28.441271 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.441214    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 08 03:27:28.442202 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.442185    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 08 03:27:28.451366 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.451340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rrf86"]
Jul 08 03:27:28.569673 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.569634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgv4w\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-kube-api-access-fgv4w\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.569673 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.569677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.670737 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.670705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgv4w\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-kube-api-access-fgv4w\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.670737 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.670741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.686096 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.686070    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgv4w\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-kube-api-access-fgv4w\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.687199 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.687124    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88d3f156-b68f-460e-abf7-71a6b76996f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rrf86\" (UID: \"88d3f156-b68f-460e-abf7-71a6b76996f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.747103 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.747070    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:28.875401 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:28.875371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rrf86"]
Jul 08 03:27:28.878313 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:28.878277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88d3f156_b68f_460e_abf7_71a6b76996f0.slice/crio-0ac3c678ea1600554b8bc562ebe079d205e5e63fd418a96acda03196524de091 WatchSource:0}: Error finding container 0ac3c678ea1600554b8bc562ebe079d205e5e63fd418a96acda03196524de091: Status 404 returned error can't find the container with id 0ac3c678ea1600554b8bc562ebe079d205e5e63fd418a96acda03196524de091
Jul 08 03:27:29.081798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:29.081762    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86" event={"ID":"88d3f156-b68f-460e-abf7-71a6b76996f0","Type":"ContainerStarted","Data":"0ac3c678ea1600554b8bc562ebe079d205e5e63fd418a96acda03196524de091"}
Jul 08 03:27:30.682619 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.682569    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"]
Jul 08 03:27:30.685058 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.685033    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.688375 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.688351    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-957gx\""
Jul 08 03:27:30.694550 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.694523    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"]
Jul 08 03:27:30.790131 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.790094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.790305 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.790194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52cfb\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-kube-api-access-52cfb\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.891184 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.891118    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52cfb\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-kube-api-access-52cfb\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.891348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.891208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.900352 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.900316    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.901321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.901286    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52cfb\" (UniqueName: \"kubernetes.io/projected/4c5406f5-afe4-475c-9fb6-4d227708f258-kube-api-access-52cfb\") pod \"cert-manager-cainjector-559cdd8c87-qmsks\" (UID: \"4c5406f5-afe4-475c-9fb6-4d227708f258\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:30.996969 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:30.996895    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"
Jul 08 03:27:31.959393 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:31.959368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-qmsks"]
Jul 08 03:27:31.960921 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:31.960893    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c5406f5_afe4_475c_9fb6_4d227708f258.slice/crio-54a751c6d7333015ac5f63959a95617c10bdee6f9bec8c51ccf16930b56098ae WatchSource:0}: Error finding container 54a751c6d7333015ac5f63959a95617c10bdee6f9bec8c51ccf16930b56098ae: Status 404 returned error can't find the container with id 54a751c6d7333015ac5f63959a95617c10bdee6f9bec8c51ccf16930b56098ae
Jul 08 03:27:32.098185 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.098077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks" event={"ID":"4c5406f5-afe4-475c-9fb6-4d227708f258","Type":"ContainerStarted","Data":"7752a4d8a057bd6177ea69d25f71951afe1c0de6750825581c501e0bf3617968"}
Jul 08 03:27:32.098185 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.098126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks" event={"ID":"4c5406f5-afe4-475c-9fb6-4d227708f258","Type":"ContainerStarted","Data":"54a751c6d7333015ac5f63959a95617c10bdee6f9bec8c51ccf16930b56098ae"}
Jul 08 03:27:32.099506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.099481    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86" event={"ID":"88d3f156-b68f-460e-abf7-71a6b76996f0","Type":"ContainerStarted","Data":"eebb4ae91c3dfa27b35500bef4b68d80efe1fb1d982520fad45e83340fd03ca8"}
Jul 08 03:27:32.099612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.099596    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:32.115964 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.115917    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-qmsks" podStartSLOduration=2.115901331 podStartE2EDuration="2.115901331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:27:32.113697237 +0000 UTC m=+463.222043178" watchObservedRunningTime="2026-07-08 03:27:32.115901331 +0000 UTC m=+463.224247264"
Jul 08 03:27:32.130777 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:32.130721    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86" podStartSLOduration=1.144889102 podStartE2EDuration="4.130700503s" totalImagesPullingTime="2.985811401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:27:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:27:32.130238589 +0000 UTC m=+463.238584530" watchObservedRunningTime="2026-07-08 03:27:32.130700503 +0000 UTC m=+463.239046444"
Jul 08 03:27:38.106408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:38.106378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rrf86"
Jul 08 03:27:41.694711 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.694674    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"]
Jul 08 03:27:41.697856 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.697836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.700379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.700358    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 08 03:27:41.700498 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.700421    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7wtxf\""
Jul 08 03:27:41.700498 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.700456    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 08 03:27:41.707033 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.707009    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"]
Jul 08 03:27:41.790013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.789978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.790222 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.790023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.790222 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.790165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whv5k\" (UniqueName: \"kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.891323 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.891281    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.891465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.891332    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.891465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.891391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whv5k\" (UniqueName: \"kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.891687 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.891666    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.891722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.891690    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:41.900227 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:41.900199    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-whv5k\" (UniqueName: \"kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:42.007682 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:42.007579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:42.143472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:42.143362    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"]
Jul 08 03:27:42.146369 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:42.146338    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17a6c0a4_f584_4198_8019_db85ee172c97.slice/crio-3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad WatchSource:0}: Error finding container 3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad: Status 404 returned error can't find the container with id 3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad
Jul 08 03:27:43.148733 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:43.148698    2563 generic.go:356] "Generic (PLEG): container finished" podID="17a6c0a4-f584-4198-8019-db85ee172c97" containerID="75f9a98072a0e99ec1baa959e8b15a5a677baa3c7d68c8ae4e44bd07d6b5671f" exitCode=0
Jul 08 03:27:43.148733 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:43.148735    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx" event={"ID":"17a6c0a4-f584-4198-8019-db85ee172c97","Type":"ContainerDied","Data":"75f9a98072a0e99ec1baa959e8b15a5a677baa3c7d68c8ae4e44bd07d6b5671f"}
Jul 08 03:27:43.149127 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:43.148771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx" event={"ID":"17a6c0a4-f584-4198-8019-db85ee172c97","Type":"ContainerStarted","Data":"3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad"}
Jul 08 03:27:46.165549 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:46.165516    2563 generic.go:356] "Generic (PLEG): container finished" podID="17a6c0a4-f584-4198-8019-db85ee172c97" containerID="5804b45d50179f5e0fc9aa3979e6923bb687df127b17adf479e28bd679c31245" exitCode=0
Jul 08 03:27:46.165930 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:46.165563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx" event={"ID":"17a6c0a4-f584-4198-8019-db85ee172c97","Type":"ContainerDied","Data":"5804b45d50179f5e0fc9aa3979e6923bb687df127b17adf479e28bd679c31245"}
Jul 08 03:27:47.171420 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:47.171384    2563 generic.go:356] "Generic (PLEG): container finished" podID="17a6c0a4-f584-4198-8019-db85ee172c97" containerID="e54d2b8e317aa9621fd27c8ebf435f6a1fd19dff3696526ef12a110e167b7a11" exitCode=0
Jul 08 03:27:47.171853 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:47.171468    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx" event={"ID":"17a6c0a4-f584-4198-8019-db85ee172c97","Type":"ContainerDied","Data":"e54d2b8e317aa9621fd27c8ebf435f6a1fd19dff3696526ef12a110e167b7a11"}
Jul 08 03:27:48.299459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.299436    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:48.450348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.450242    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whv5k\" (UniqueName: \"kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k\") pod \"17a6c0a4-f584-4198-8019-db85ee172c97\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") "
Jul 08 03:27:48.450348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.450330    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle\") pod \"17a6c0a4-f584-4198-8019-db85ee172c97\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") "
Jul 08 03:27:48.450571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.450374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util\") pod \"17a6c0a4-f584-4198-8019-db85ee172c97\" (UID: \"17a6c0a4-f584-4198-8019-db85ee172c97\") "
Jul 08 03:27:48.450739 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.450716    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle" (OuterVolumeSpecName: "bundle") pod "17a6c0a4-f584-4198-8019-db85ee172c97" (UID: "17a6c0a4-f584-4198-8019-db85ee172c97"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:27:48.452604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.452580    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k" (OuterVolumeSpecName: "kube-api-access-whv5k") pod "17a6c0a4-f584-4198-8019-db85ee172c97" (UID: "17a6c0a4-f584-4198-8019-db85ee172c97"). InnerVolumeSpecName "kube-api-access-whv5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:27:48.454837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.454817    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util" (OuterVolumeSpecName: "util") pod "17a6c0a4-f584-4198-8019-db85ee172c97" (UID: "17a6c0a4-f584-4198-8019-db85ee172c97"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:27:48.551690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.551636    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whv5k\" (UniqueName: \"kubernetes.io/projected/17a6c0a4-f584-4198-8019-db85ee172c97-kube-api-access-whv5k\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:48.551690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.551680    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:48.551690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:48.551694    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/17a6c0a4-f584-4198-8019-db85ee172c97-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:27:49.181908 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:49.181875    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx"
Jul 08 03:27:49.181908 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:49.181891    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2kvlx" event={"ID":"17a6c0a4-f584-4198-8019-db85ee172c97","Type":"ContainerDied","Data":"3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad"}
Jul 08 03:27:49.182114 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:49.181934    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3fced5c14cf1bd5912684c35e5b94093b681f7e1a302bdad19995cdbae3c7cad"
Jul 08 03:27:54.295395 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295359    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"]
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295671    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="pull"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="pull"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295696    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="extract"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295702    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="extract"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295709    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="util"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295715    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="util"
Jul 08 03:27:54.295838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.295762    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="17a6c0a4-f584-4198-8019-db85ee172c97" containerName="extract"
Jul 08 03:27:54.298363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.298343    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.300900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.300874    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-lml7h\""
Jul 08 03:27:54.301008 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.300938    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 03:27:54.301008 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.300883    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 03:27:54.307610 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.307586    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"]
Jul 08 03:27:54.401032 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.401001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qnnj\" (UniqueName: \"kubernetes.io/projected/8696c5c5-39ad-4b62-94d4-fe8967a672e2-kube-api-access-4qnnj\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.401230 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.401099    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8696c5c5-39ad-4b62-94d4-fe8967a672e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.502262 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.502206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8696c5c5-39ad-4b62-94d4-fe8967a672e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.502422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.502333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qnnj\" (UniqueName: \"kubernetes.io/projected/8696c5c5-39ad-4b62-94d4-fe8967a672e2-kube-api-access-4qnnj\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.502545 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.502526    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8696c5c5-39ad-4b62-94d4-fe8967a672e2-tmp\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.510701 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.510677    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qnnj\" (UniqueName: \"kubernetes.io/projected/8696c5c5-39ad-4b62-94d4-fe8967a672e2-kube-api-access-4qnnj\") pod \"openshift-lws-operator-fd8ccff4c-vwgnr\" (UID: \"8696c5c5-39ad-4b62-94d4-fe8967a672e2\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.608372 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.608281    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"
Jul 08 03:27:54.737426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:54.737187    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr"]
Jul 08 03:27:54.743093 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:27:54.743064    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8696c5c5_39ad_4b62_94d4_fe8967a672e2.slice/crio-9b313087c2ad852ff37f4bf148029cc569d39eb17c7fd87fca2fcf76d42dd013 WatchSource:0}: Error finding container 9b313087c2ad852ff37f4bf148029cc569d39eb17c7fd87fca2fcf76d42dd013: Status 404 returned error can't find the container with id 9b313087c2ad852ff37f4bf148029cc569d39eb17c7fd87fca2fcf76d42dd013
Jul 08 03:27:55.209738 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:55.209705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr" event={"ID":"8696c5c5-39ad-4b62-94d4-fe8967a672e2","Type":"ContainerStarted","Data":"9b313087c2ad852ff37f4bf148029cc569d39eb17c7fd87fca2fcf76d42dd013"}
Jul 08 03:27:57.219721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:57.219684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr" event={"ID":"8696c5c5-39ad-4b62-94d4-fe8967a672e2","Type":"ContainerStarted","Data":"6fef5419656d27dfc8d5f3162480e4f329b44f4b8259ff5edefd6be8b554cfd7"}
Jul 08 03:27:57.238805 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:57.238758    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-vwgnr" podStartSLOduration=1.199158516 podStartE2EDuration="3.238743015s" totalImagesPullingTime="2.039584499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:27:57.23772571 +0000 UTC m=+488.346071649" watchObservedRunningTime="2026-07-08 03:27:57.238743015 +0000 UTC m=+488.347088954"
Jul 08 03:27:59.882816 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.882785    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"]
Jul 08 03:27:59.885484 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.885467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:27:59.888101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.888077    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7wtxf\""
Jul 08 03:27:59.888243 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.888126    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 08 03:27:59.888243 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.888089    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 08 03:27:59.893818 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.893796    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"]
Jul 08 03:27:59.944402 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.944367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpjqq\" (UniqueName: \"kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:27:59.944558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.944412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:27:59.944558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:27:59.944490    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.045491 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.045450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.045680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.045543    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpjqq\" (UniqueName: \"kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.045680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.045578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.045914 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.045889    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.045980 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.045898    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.055328 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.055307    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpjqq\" (UniqueName: \"kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.195826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.195731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:00.329516 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:00.329493    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"]
Jul 08 03:28:00.332220 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:00.332192    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8ebb25d_b6dd_44c8_a3ef_be534bd05b76.slice/crio-65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba WatchSource:0}: Error finding container 65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba: Status 404 returned error can't find the container with id 65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba
Jul 08 03:28:01.238794 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:01.238759    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerID="e1129c18f4069e63cda64efa268ead7ce2e66297b115d4cdf1e4a17cebcef652" exitCode=0
Jul 08 03:28:01.239304 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:01.238824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr" event={"ID":"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76","Type":"ContainerDied","Data":"e1129c18f4069e63cda64efa268ead7ce2e66297b115d4cdf1e4a17cebcef652"}
Jul 08 03:28:01.239304 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:01.238881    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr" event={"ID":"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76","Type":"ContainerStarted","Data":"65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba"}
Jul 08 03:28:02.245668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:02.245576    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerID="6c4175f3185300439c404359c23f4af2dcf0674dae1fd0442d8da8348a785931" exitCode=0
Jul 08 03:28:02.245668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:02.245641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr" event={"ID":"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76","Type":"ContainerDied","Data":"6c4175f3185300439c404359c23f4af2dcf0674dae1fd0442d8da8348a785931"}
Jul 08 03:28:03.251979 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:03.251943    2563 generic.go:356] "Generic (PLEG): container finished" podID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerID="89fb862eb748762aafaf44656274949088f49881b1b476573af7acf38b652fe7" exitCode=0
Jul 08 03:28:03.252411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:03.252041    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr" event={"ID":"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76","Type":"ContainerDied","Data":"89fb862eb748762aafaf44656274949088f49881b1b476573af7acf38b652fe7"}
Jul 08 03:28:04.413768 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.413734    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:04.481316 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.481284    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle\") pod \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") "
Jul 08 03:28:04.481316 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.481323    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util\") pod \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") "
Jul 08 03:28:04.481598 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.481367    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpjqq\" (UniqueName: \"kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq\") pod \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\" (UID: \"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76\") "
Jul 08 03:28:04.481984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.481954    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle" (OuterVolumeSpecName: "bundle") pod "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" (UID: "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:28:04.483603 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.483574    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq" (OuterVolumeSpecName: "kube-api-access-wpjqq") pod "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" (UID: "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76"). InnerVolumeSpecName "kube-api-access-wpjqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:28:04.486906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.486867    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util" (OuterVolumeSpecName: "util") pod "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" (UID: "f8ebb25d-b6dd-44c8-a3ef-be534bd05b76"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:28:04.582760 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.582727    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:04.582760 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.582754    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:04.582760 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:04.582766    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpjqq\" (UniqueName: \"kubernetes.io/projected/f8ebb25d-b6dd-44c8-a3ef-be534bd05b76-kube-api-access-wpjqq\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:05.262586 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:05.262557    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr"
Jul 08 03:28:05.262756 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:05.262584    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afkc2dr" event={"ID":"f8ebb25d-b6dd-44c8-a3ef-be534bd05b76","Type":"ContainerDied","Data":"65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba"}
Jul 08 03:28:05.262756 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:05.262652    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65e2d6eca030ede125c5f9d2eb8d9b4ab8589cfeb4c1fefd6d86efd4e930f4ba"
Jul 08 03:28:15.672898 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.672857    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"]
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673290    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="extract"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673309    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="extract"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673323    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="pull"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673331    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="pull"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673346    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="util"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673356    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="util"
Jul 08 03:28:15.673475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.673447    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8ebb25d-b6dd-44c8-a3ef-be534bd05b76" containerName="extract"
Jul 08 03:28:15.679447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.679426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.681987 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.681958    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 03:28:15.682222 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.682203    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 03:28:15.682329 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.682290    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 03:28:15.682329 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.682290    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-9xhxh\""
Jul 08 03:28:15.682565 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.682549    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 03:28:15.696038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.696017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"]
Jul 08 03:28:15.781965 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.781932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.781965 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.781969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k74qg\" (UniqueName: \"kubernetes.io/projected/c87e613a-f280-4a78-8f4b-fd8579bd264e-kube-api-access-k74qg\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.782258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.781999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.782258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.782098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.782258 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.782127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.882938 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.882898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.882938 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.882940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k74qg\" (UniqueName: \"kubernetes.io/projected/c87e613a-f280-4a78-8f4b-fd8579bd264e-kube-api-access-k74qg\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.883208 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.882969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.883208 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.883005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.883208 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.883029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.883447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.883420    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-manifests\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.885366 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.885342    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c87e613a-f280-4a78-8f4b-fd8579bd264e-tmp\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.885519 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.885503    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.885559 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.885535    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c87e613a-f280-4a78-8f4b-fd8579bd264e-webhook-cert\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.898984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.898958    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k74qg\" (UniqueName: \"kubernetes.io/projected/c87e613a-f280-4a78-8f4b-fd8579bd264e-kube-api-access-k74qg\") pod \"opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c\" (UID: \"c87e613a-f280-4a78-8f4b-fd8579bd264e\") " pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:15.990387 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:15.990308    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:16.129230 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:16.129203    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"]
Jul 08 03:28:16.131106 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:16.131083    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc87e613a_f280_4a78_8f4b_fd8579bd264e.slice/crio-16b56ea2ea64fbee22b3d8f80a676748f2ddf7820de160241796dbba07399a0f WatchSource:0}: Error finding container 16b56ea2ea64fbee22b3d8f80a676748f2ddf7820de160241796dbba07399a0f: Status 404 returned error can't find the container with id 16b56ea2ea64fbee22b3d8f80a676748f2ddf7820de160241796dbba07399a0f
Jul 08 03:28:16.312724 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:16.312683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c" event={"ID":"c87e613a-f280-4a78-8f4b-fd8579bd264e","Type":"ContainerStarted","Data":"16b56ea2ea64fbee22b3d8f80a676748f2ddf7820de160241796dbba07399a0f"}
Jul 08 03:28:20.339525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:20.339488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c" event={"ID":"c87e613a-f280-4a78-8f4b-fd8579bd264e","Type":"ContainerStarted","Data":"7f2d58209c8cc8c39c2512498e20db0c40b364956950e80c952fd92640cc6cd6"}
Jul 08 03:28:21.345834 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:21.345799    2563 generic.go:356] "Generic (PLEG): container finished" podID="c87e613a-f280-4a78-8f4b-fd8579bd264e" containerID="7f2d58209c8cc8c39c2512498e20db0c40b364956950e80c952fd92640cc6cd6" exitCode=0
Jul 08 03:28:21.346229 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:21.345879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c" event={"ID":"c87e613a-f280-4a78-8f4b-fd8579bd264e","Type":"ContainerDied","Data":"7f2d58209c8cc8c39c2512498e20db0c40b364956950e80c952fd92640cc6cd6"}
Jul 08 03:28:22.352753 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:22.352716    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c" event={"ID":"c87e613a-f280-4a78-8f4b-fd8579bd264e","Type":"ContainerStarted","Data":"cb63d186521612957eee5010f34fc1454c151c945fc74aea270b6ee72ea47bfb"}
Jul 08 03:28:22.353167 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:22.352835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:22.381896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:22.381847    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c" podStartSLOduration=2.560426953 podStartE2EDuration="7.381828408s" totalImagesPullingTime="3.762875348s" totalInitContainerRuntime="1.058526107s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:15 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:22.380926427 +0000 UTC m=+513.489272386" watchObservedRunningTime="2026-07-08 03:28:22.381828408 +0000 UTC m=+513.490174340"
Jul 08 03:28:33.359992 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:33.359963    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c"
Jul 08 03:28:35.508252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.508215    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"]
Jul 08 03:28:35.518263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.518231    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.521956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.521934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jds27\""
Jul 08 03:28:35.523282 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.523257    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 03:28:35.524309 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.524293    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 03:28:35.529296 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.529275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"]
Jul 08 03:28:35.652429 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.652394    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612f803-f607-4d02-bc14-0ba3328c8d03-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.652429 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.652434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x59h2\" (UniqueName: \"kubernetes.io/projected/2612f803-f607-4d02-bc14-0ba3328c8d03-kube-api-access-x59h2\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.652624 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.652472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2612f803-f607-4d02-bc14-0ba3328c8d03-tmp\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.753387 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.753351    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612f803-f607-4d02-bc14-0ba3328c8d03-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.753387 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.753391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x59h2\" (UniqueName: \"kubernetes.io/projected/2612f803-f607-4d02-bc14-0ba3328c8d03-kube-api-access-x59h2\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.753595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.753430    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2612f803-f607-4d02-bc14-0ba3328c8d03-tmp\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.755882 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.755856    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2612f803-f607-4d02-bc14-0ba3328c8d03-tmp\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.756130 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.756107    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612f803-f607-4d02-bc14-0ba3328c8d03-tls-certs\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.777722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.777657    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bl9cn"]
Jul 08 03:28:35.780091 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.780067    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x59h2\" (UniqueName: \"kubernetes.io/projected/2612f803-f607-4d02-bc14-0ba3328c8d03-kube-api-access-x59h2\") pod \"kube-auth-proxy-69556cfdfb-b7drh\" (UID: \"2612f803-f607-4d02-bc14-0ba3328c8d03\") " pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.781352 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.781338    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:35.788786 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.788764    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 08 03:28:35.790533 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.790515    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-qlvlr\""
Jul 08 03:28:35.803063 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.803041    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bl9cn"]
Jul 08 03:28:35.827787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.827757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"
Jul 08 03:28:35.956241 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.956211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rbps\" (UniqueName: \"kubernetes.io/projected/115aced2-45b7-4e4b-9ddb-4424117fac5b-kube-api-access-8rbps\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:35.956422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.956265    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:35.982896 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:35.982853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh"]
Jul 08 03:28:35.984889 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:35.984859    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2612f803_f607_4d02_bc14_0ba3328c8d03.slice/crio-58202da8e66c607ca85d159ec5f79fcfc9b3a1caf80c56929a63bb784a67de6a WatchSource:0}: Error finding container 58202da8e66c607ca85d159ec5f79fcfc9b3a1caf80c56929a63bb784a67de6a: Status 404 returned error can't find the container with id 58202da8e66c607ca85d159ec5f79fcfc9b3a1caf80c56929a63bb784a67de6a
Jul 08 03:28:36.057635 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.057605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rbps\" (UniqueName: \"kubernetes.io/projected/115aced2-45b7-4e4b-9ddb-4424117fac5b-kube-api-access-8rbps\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.057830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.057651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.057830 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:28:36.057780    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 08 03:28:36.057946 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:28:36.057839    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert podName:115aced2-45b7-4e4b-9ddb-4424117fac5b nodeName:}" failed. No retries permitted until 2026-07-08 03:28:36.557822955 +0000 UTC m=+527.666168873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert") pod "odh-model-controller-7998c67dc4-bl9cn" (UID: "115aced2-45b7-4e4b-9ddb-4424117fac5b") : secret "odh-model-controller-webhook-cert" not found
Jul 08 03:28:36.080942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.080910    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rbps\" (UniqueName: \"kubernetes.io/projected/115aced2-45b7-4e4b-9ddb-4424117fac5b-kube-api-access-8rbps\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.418093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.418011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh" event={"ID":"2612f803-f607-4d02-bc14-0ba3328c8d03","Type":"ContainerStarted","Data":"58202da8e66c607ca85d159ec5f79fcfc9b3a1caf80c56929a63bb784a67de6a"}
Jul 08 03:28:36.562516 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.562486    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.565068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.565035    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/115aced2-45b7-4e4b-9ddb-4424117fac5b-cert\") pod \"odh-model-controller-7998c67dc4-bl9cn\" (UID: \"115aced2-45b7-4e4b-9ddb-4424117fac5b\") " pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.694481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.694388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:36.877609 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.877577    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bl9cn"]
Jul 08 03:28:36.879119 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:36.879090    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod115aced2_45b7_4e4b_9ddb_4424117fac5b.slice/crio-59ad4d6421e8ac2d16eb9800a8ba082b170f6f2d3b4458939dedc4dc0c60b3ec WatchSource:0}: Error finding container 59ad4d6421e8ac2d16eb9800a8ba082b170f6f2d3b4458939dedc4dc0c60b3ec: Status 404 returned error can't find the container with id 59ad4d6421e8ac2d16eb9800a8ba082b170f6f2d3b4458939dedc4dc0c60b3ec
Jul 08 03:28:36.895073 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.895045    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vwb5b"]
Jul 08 03:28:36.899879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.899852    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:36.905884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.905861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-4rh92\""
Jul 08 03:28:36.906467 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.906294    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 08 03:28:36.909818 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:36.909785    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vwb5b"]
Jul 08 03:28:37.066986 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.066956    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm2wl\" (UniqueName: \"kubernetes.io/projected/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-kube-api-access-wm2wl\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.067194 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.067011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-tls-certs\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.104780 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.104744    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:37.108309 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.108283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.113165 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.113117    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 08 03:28:37.114118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.114070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 08 03:28:37.114264 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.114245    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 03:28:37.114335 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.114234    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 08 03:28:37.114373 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.114334    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-9pg5h\""
Jul 08 03:28:37.130660 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.130632    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:37.167740 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.167710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm2wl\" (UniqueName: \"kubernetes.io/projected/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-kube-api-access-wm2wl\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.167926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.167753    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-tls-certs\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.170669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.170645    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-tls-certs\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.183424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.183399    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm2wl\" (UniqueName: \"kubernetes.io/projected/583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa-kube-api-access-wm2wl\") pod \"model-serving-api-54d89bd5fc-vwb5b\" (UID: \"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa\") " pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.214052 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.214013    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:37.269087 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cksmb\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269087 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269087 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269115    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269207    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.269436 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.269260    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.370775 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.370733    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.370960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.370822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.370960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.370852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cksmb\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.370960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.370921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.371134 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.370969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.371134 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.371001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.371134 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.371032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.371898 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.371873    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.374257 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.374218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.374947 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.374805    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.375261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.375237    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.375355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.375248    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.377176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.377028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-vwb5b"]
Jul 08 03:28:37.380510 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:37.380482    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod583ae9b9_5b3b_4ccc_b932_9ddcd9daa3fa.slice/crio-3edabc79c75e3c940221e5fecfdfff454f248b93faa541a78e9f4c2c1f0119e5 WatchSource:0}: Error finding container 3edabc79c75e3c940221e5fecfdfff454f248b93faa541a78e9f4c2c1f0119e5: Status 404 returned error can't find the container with id 3edabc79c75e3c940221e5fecfdfff454f248b93faa541a78e9f4c2c1f0119e5
Jul 08 03:28:37.381034 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.381000    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.387115 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.381584    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cksmb\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb\") pod \"istiod-openshift-gateway-7574459656-fhx4m\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.419577 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.419547    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:37.423773 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.423736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b" event={"ID":"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa","Type":"ContainerStarted","Data":"3edabc79c75e3c940221e5fecfdfff454f248b93faa541a78e9f4c2c1f0119e5"}
Jul 08 03:28:37.425317 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.425290    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn" event={"ID":"115aced2-45b7-4e4b-9ddb-4424117fac5b","Type":"ContainerStarted","Data":"59ad4d6421e8ac2d16eb9800a8ba082b170f6f2d3b4458939dedc4dc0c60b3ec"}
Jul 08 03:28:37.587361 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:37.587331    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:37.590134 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:37.590098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23cb4822_0ab7_487a_b307_dbc057badd6e.slice/crio-15c492a063a8f6697bf95073fbd3bb5fca6fdd5c0c8133f52411b081a30d8cde WatchSource:0}: Error finding container 15c492a063a8f6697bf95073fbd3bb5fca6fdd5c0c8133f52411b081a30d8cde: Status 404 returned error can't find the container with id 15c492a063a8f6697bf95073fbd3bb5fca6fdd5c0c8133f52411b081a30d8cde
Jul 08 03:28:38.434060 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:38.434018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" event={"ID":"23cb4822-0ab7-487a-b307-dbc057badd6e","Type":"ContainerStarted","Data":"15c492a063a8f6697bf95073fbd3bb5fca6fdd5c0c8133f52411b081a30d8cde"}
Jul 08 03:28:41.734890 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:41.734856    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:28:41.735224 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:41.734931    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:28:42.458228 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.458184    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh" event={"ID":"2612f803-f607-4d02-bc14-0ba3328c8d03","Type":"ContainerStarted","Data":"c8c90c6fe001f9b1fa42f588d8da92013ecddf90432996888f03496ddd103acb"}
Jul 08 03:28:42.459850 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.459818    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" event={"ID":"23cb4822-0ab7-487a-b307-dbc057badd6e","Type":"ContainerStarted","Data":"2b9fcae805eab54be6877f09a2c080331fd7d2d2090ce867b4e8ca45afd8bd84"}
Jul 08 03:28:42.460025 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.460008    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:42.461541 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.461503    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b" event={"ID":"583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa","Type":"ContainerStarted","Data":"e517b1eeedeca0f8418c0cd7951abd8fd6359c733c0041ce48028b43d0ab6623"}
Jul 08 03:28:42.461825 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.461734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:42.462078 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.462056    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fhx4m container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 03:28:42.462191 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.462102    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 03:28:42.463035 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.463013    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn" event={"ID":"115aced2-45b7-4e4b-9ddb-4424117fac5b","Type":"ContainerStarted","Data":"e4609267b4de09a5d426c35a3d3ccd9101dde02641808f14aae0dcf93c58a9c4"}
Jul 08 03:28:42.463169 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.463080    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:42.477353 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.477301    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69556cfdfb-b7drh" podStartSLOduration=1.798125197 podStartE2EDuration="7.477281876s" totalImagesPullingTime="5.679156679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:42.475621553 +0000 UTC m=+533.583967494" watchObservedRunningTime="2026-07-08 03:28:42.477281876 +0000 UTC m=+533.585627815"
Jul 08 03:28:42.499640 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.499568    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" podStartSLOduration=1.357682522 podStartE2EDuration="5.499547362s" totalImagesPullingTime="4.14186484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:42.497355214 +0000 UTC m=+533.605701157" watchObservedRunningTime="2026-07-08 03:28:42.499547362 +0000 UTC m=+533.607893304"
Jul 08 03:28:42.517563 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.517507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b" podStartSLOduration=2.184095361 podStartE2EDuration="6.517490938s" totalImagesPullingTime="4.333395577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:42.515601208 +0000 UTC m=+533.623947143" watchObservedRunningTime="2026-07-08 03:28:42.517490938 +0000 UTC m=+533.625836877"
Jul 08 03:28:42.534461 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:42.534410    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn" podStartSLOduration=2.66819412 podStartE2EDuration="7.534393345s" totalImagesPullingTime="4.866199225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:42.532210828 +0000 UTC m=+533.640556768" watchObservedRunningTime="2026-07-08 03:28:42.534393345 +0000 UTC m=+533.642739285"
Jul 08 03:28:43.468721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:43.468695    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:46.588302 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.588269    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"]
Jul 08 03:28:46.591799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.591778    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.596717 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.596696    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 08 03:28:46.599506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.599482    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-4dmkb\""
Jul 08 03:28:46.621024 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.620997    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"]
Jul 08 03:28:46.624479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.624457    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"]
Jul 08 03:28:46.624593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.624566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.631512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.631493    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 08 03:28:46.631826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.631700    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p2b8f\""
Jul 08 03:28:46.658504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.658471    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"]
Jul 08 03:28:46.665988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.665958    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7hl8\" (UniqueName: \"kubernetes.io/projected/45057a34-5fa4-44bf-bfa3-68afa7748696-kube-api-access-v7hl8\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.666128 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.666036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6xv5\" (UniqueName: \"kubernetes.io/projected/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-kube-api-access-t6xv5\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.666128 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.666090    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45057a34-5fa4-44bf-bfa3-68afa7748696-cert\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.666236 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.666219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.767115 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.767074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45057a34-5fa4-44bf-bfa3-68afa7748696-cert\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.767343 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.767165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.767343 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.767231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7hl8\" (UniqueName: \"kubernetes.io/projected/45057a34-5fa4-44bf-bfa3-68afa7748696-kube-api-access-v7hl8\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.767343 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.767293    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6xv5\" (UniqueName: \"kubernetes.io/projected/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-kube-api-access-t6xv5\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.767343 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:28:46.767296    2563 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 08 03:28:46.767562 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:28:46.767369    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert podName:c4f1f2e3-aa65-48e1-ad25-c2f301658ed1 nodeName:}" failed. No retries permitted until 2026-07-08 03:28:47.267347376 +0000 UTC m=+538.375693297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert") pod "llmisvc-controller-manager-777c6bd7d5-rr99j" (UID: "c4f1f2e3-aa65-48e1-ad25-c2f301658ed1") : secret "llmisvc-webhook-server-cert" not found
Jul 08 03:28:46.769712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.769680    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/45057a34-5fa4-44bf-bfa3-68afa7748696-cert\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.781247 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.781223    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7hl8\" (UniqueName: \"kubernetes.io/projected/45057a34-5fa4-44bf-bfa3-68afa7748696-kube-api-access-v7hl8\") pod \"kserve-controller-manager-d6f4b65b8-zdd8t\" (UID: \"45057a34-5fa4-44bf-bfa3-68afa7748696\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:46.793285 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.793262    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6xv5\" (UniqueName: \"kubernetes.io/projected/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-kube-api-access-t6xv5\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:46.916666 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:46.916575    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:47.080681 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:47.080649    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45057a34_5fa4_44bf_bfa3_68afa7748696.slice/crio-6a805c617e87ed794299b48e964c445d62354e6104e5ce2e34b3135d763e0a22 WatchSource:0}: Error finding container 6a805c617e87ed794299b48e964c445d62354e6104e5ce2e34b3135d763e0a22: Status 404 returned error can't find the container with id 6a805c617e87ed794299b48e964c445d62354e6104e5ce2e34b3135d763e0a22
Jul 08 03:28:47.081015 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.080991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"]
Jul 08 03:28:47.272495 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.272391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:47.275228 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.275194    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c4f1f2e3-aa65-48e1-ad25-c2f301658ed1-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-rr99j\" (UID: \"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:47.490455 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.490421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t" event={"ID":"45057a34-5fa4-44bf-bfa3-68afa7748696","Type":"ContainerStarted","Data":"6a805c617e87ed794299b48e964c445d62354e6104e5ce2e34b3135d763e0a22"}
Jul 08 03:28:47.539371 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.539289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:47.677399 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:47.677377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"]
Jul 08 03:28:47.679858 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:28:47.679825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc4f1f2e3_aa65_48e1_ad25_c2f301658ed1.slice/crio-f786a3c2258bfaf93300dbc6e0501de9a17b4a9f49142bf1ee2d6845e8edb706 WatchSource:0}: Error finding container f786a3c2258bfaf93300dbc6e0501de9a17b4a9f49142bf1ee2d6845e8edb706: Status 404 returned error can't find the container with id f786a3c2258bfaf93300dbc6e0501de9a17b4a9f49142bf1ee2d6845e8edb706
Jul 08 03:28:48.498197 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:48.497971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j" event={"ID":"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1","Type":"ContainerStarted","Data":"f786a3c2258bfaf93300dbc6e0501de9a17b4a9f49142bf1ee2d6845e8edb706"}
Jul 08 03:28:48.925890 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:48.925823    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:48.926314 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:48.926037    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery" containerID="cri-o://2b9fcae805eab54be6877f09a2c080331fd7d2d2090ce867b4e8ca45afd8bd84" gracePeriod=30
Jul 08 03:28:49.506124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.506090    2563 generic.go:356] "Generic (PLEG): container finished" podID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerID="2b9fcae805eab54be6877f09a2c080331fd7d2d2090ce867b4e8ca45afd8bd84" exitCode=0
Jul 08 03:28:49.506380 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.506131    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" event={"ID":"23cb4822-0ab7-487a-b307-dbc057badd6e","Type":"ContainerDied","Data":"2b9fcae805eab54be6877f09a2c080331fd7d2d2090ce867b4e8ca45afd8bd84"}
Jul 08 03:28:49.731801 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.731771    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:49.799404 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799382    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799424    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cksmb\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799443    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799681    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.799802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.799735    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap\") pod \"23cb4822-0ab7-487a-b307-dbc057badd6e\" (UID: \"23cb4822-0ab7-487a-b307-dbc057badd6e\") "
Jul 08 03:28:49.800565 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.800530    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:28:49.801910 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.801869    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:28:49.802000 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.801882    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs" (OuterVolumeSpecName: "local-certs") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:28:49.802240 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.802219    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token" (OuterVolumeSpecName: "istio-token") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:28:49.802305 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.802235    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb" (OuterVolumeSpecName: "kube-api-access-cksmb") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "kube-api-access-cksmb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:28:49.802348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.802332    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts" (OuterVolumeSpecName: "cacerts") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:28:49.802426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.802408    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "23cb4822-0ab7-487a-b307-dbc057badd6e" (UID: "23cb4822-0ab7-487a-b307-dbc057badd6e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:28:49.900831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900792    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-token\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.900831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900831    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-dns-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.901057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900846    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-kubeconfig\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.901057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900860    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/23cb4822-0ab7-487a-b307-dbc057badd6e-istio-csr-ca-configmap\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.901057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900874    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/23cb4822-0ab7-487a-b307-dbc057badd6e-cacerts\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.901057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900887    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cksmb\" (UniqueName: \"kubernetes.io/projected/23cb4822-0ab7-487a-b307-dbc057badd6e-kube-api-access-cksmb\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:49.901057 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:49.900902    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/23cb4822-0ab7-487a-b307-dbc057badd6e-local-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:28:50.511873 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.511852    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"
Jul 08 03:28:50.512270 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.511847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" event={"ID":"23cb4822-0ab7-487a-b307-dbc057badd6e","Type":"ContainerDied","Data":"15c492a063a8f6697bf95073fbd3bb5fca6fdd5c0c8133f52411b081a30d8cde"}
Jul 08 03:28:50.512270 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.512014    2563 scope.go:117] "RemoveContainer" containerID="2b9fcae805eab54be6877f09a2c080331fd7d2d2090ce867b4e8ca45afd8bd84"
Jul 08 03:28:50.513695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.513611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t" event={"ID":"45057a34-5fa4-44bf-bfa3-68afa7748696","Type":"ContainerStarted","Data":"50bd83fd81e20ceb37aa58c3d31270e361d5abe7170668a35d724c164f0d70a4"}
Jul 08 03:28:50.513813 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.513758    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:28:50.537341 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.537302    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t" podStartSLOduration=1.849002636 podStartE2EDuration="4.53728609s" totalImagesPullingTime="2.688283454s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:50.535939597 +0000 UTC m=+541.644285537" watchObservedRunningTime="2026-07-08 03:28:50.53728609 +0000 UTC m=+541.645632030"
Jul 08 03:28:50.552943 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.552918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:50.557676 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:50.557650    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m"]
Jul 08 03:28:51.385430 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:51.385395    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" path="/var/lib/kubelet/pods/23cb4822-0ab7-487a-b307-dbc057badd6e/volumes"
Jul 08 03:28:51.520112 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:51.520075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j" event={"ID":"c4f1f2e3-aa65-48e1-ad25-c2f301658ed1","Type":"ContainerStarted","Data":"0fa70a7d6d547c9955adee4aecc551033d1236c4393455513cdafde4bf4d7fe9"}
Jul 08 03:28:51.520513 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:51.520162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:28:51.555536 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:51.555467    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j" podStartSLOduration=2.7331774920000003 podStartE2EDuration="5.555445206s" totalImagesPullingTime="2.822267714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:28:51.554090568 +0000 UTC m=+542.662436507" watchObservedRunningTime="2026-07-08 03:28:51.555445206 +0000 UTC m=+542.663791150"
Jul 08 03:28:53.470068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:53.470041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bl9cn"
Jul 08 03:28:53.471962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:53.471944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-vwb5b"
Jul 08 03:28:54.469165 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:54.469098    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fhx4m container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.32:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 03:28:54.469375 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:28:54.469197    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fhx4m" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery" probeResult="failure" output="Get \"http://10.133.0.32:8080/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 03:29:21.525918 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:21.525886    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-zdd8t"
Jul 08 03:29:22.528410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:22.528382    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-rr99j"
Jul 08 03:29:36.550945 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.550902    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:36.551342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.551269    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery"
Jul 08 03:29:36.551342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.551282    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery"
Jul 08 03:29:36.551415 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.551353    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="23cb4822-0ab7-487a-b307-dbc057badd6e" containerName="discovery"
Jul 08 03:29:36.560614 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.560588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:36.563387 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.563356    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 03:29:36.563538 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.563505    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 03:29:36.563612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.563572    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-7l82t\""
Jul 08 03:29:36.565434 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.565409    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:36.616482 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.616451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swd6r\" (UniqueName: \"kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r\") pod \"kuadrant-operator-catalog-bbmj7\" (UID: \"7e06825f-536f-4150-9c16-6137c7394214\") " pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:36.717007 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.716971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swd6r\" (UniqueName: \"kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r\") pod \"kuadrant-operator-catalog-bbmj7\" (UID: \"7e06825f-536f-4150-9c16-6137c7394214\") " pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:36.725157 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.725107    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swd6r\" (UniqueName: \"kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r\") pod \"kuadrant-operator-catalog-bbmj7\" (UID: \"7e06825f-536f-4150-9c16-6137c7394214\") " pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:36.873265 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.873172    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:36.918589 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:36.918554    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:37.003888 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.003860    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:37.005574 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:37.005548    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e06825f_536f_4150_9c16_6137c7394214.slice/crio-e96c9329ab9d2a9c2e181e8521d75f66c5504fcf78dc4cf530a18ea7840caac5 WatchSource:0}: Error finding container e96c9329ab9d2a9c2e181e8521d75f66c5504fcf78dc4cf530a18ea7840caac5: Status 404 returned error can't find the container with id e96c9329ab9d2a9c2e181e8521d75f66c5504fcf78dc4cf530a18ea7840caac5
Jul 08 03:29:37.122777 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.122749    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mnhl"]
Jul 08 03:29:37.125433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.125391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:37.134200 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.134179    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mnhl"]
Jul 08 03:29:37.221496 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.221466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mb7xn\" (UniqueName: \"kubernetes.io/projected/8e852069-5b7e-46aa-9171-d3cd658d5a40-kube-api-access-mb7xn\") pod \"kuadrant-operator-catalog-2mnhl\" (UID: \"8e852069-5b7e-46aa-9171-d3cd658d5a40\") " pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:37.322197 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.322166    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mb7xn\" (UniqueName: \"kubernetes.io/projected/8e852069-5b7e-46aa-9171-d3cd658d5a40-kube-api-access-mb7xn\") pod \"kuadrant-operator-catalog-2mnhl\" (UID: \"8e852069-5b7e-46aa-9171-d3cd658d5a40\") " pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:37.332524 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.332496    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mb7xn\" (UniqueName: \"kubernetes.io/projected/8e852069-5b7e-46aa-9171-d3cd658d5a40-kube-api-access-mb7xn\") pod \"kuadrant-operator-catalog-2mnhl\" (UID: \"8e852069-5b7e-46aa-9171-d3cd658d5a40\") " pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:37.435994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.435897    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:37.580414 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.580390    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2mnhl"]
Jul 08 03:29:37.581595 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:37.581570    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e852069_5b7e_46aa_9171_d3cd658d5a40.slice/crio-1784cad7d9d01ba74f5770149326cc0d17d8624ba6c3119e89d4e318e86eda8d WatchSource:0}: Error finding container 1784cad7d9d01ba74f5770149326cc0d17d8624ba6c3119e89d4e318e86eda8d: Status 404 returned error can't find the container with id 1784cad7d9d01ba74f5770149326cc0d17d8624ba6c3119e89d4e318e86eda8d
Jul 08 03:29:37.744273 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.744179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl" event={"ID":"8e852069-5b7e-46aa-9171-d3cd658d5a40","Type":"ContainerStarted","Data":"1784cad7d9d01ba74f5770149326cc0d17d8624ba6c3119e89d4e318e86eda8d"}
Jul 08 03:29:37.745429 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:37.745401    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" event={"ID":"7e06825f-536f-4150-9c16-6137c7394214","Type":"ContainerStarted","Data":"e96c9329ab9d2a9c2e181e8521d75f66c5504fcf78dc4cf530a18ea7840caac5"}
Jul 08 03:29:39.757822 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:39.757781    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl" event={"ID":"8e852069-5b7e-46aa-9171-d3cd658d5a40","Type":"ContainerStarted","Data":"e82abf8c1cc4862cc251a3c7eaece1638b84e188481eef951014db96bdf9e224"}
Jul 08 03:29:39.759308 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:39.759283    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" event={"ID":"7e06825f-536f-4150-9c16-6137c7394214","Type":"ContainerStarted","Data":"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"}
Jul 08 03:29:39.759428 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:39.759329    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" podUID="7e06825f-536f-4150-9c16-6137c7394214" containerName="registry-server" containerID="cri-o://afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547" gracePeriod=2
Jul 08 03:29:39.775038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:39.774994    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl" podStartSLOduration=1.362547146 podStartE2EDuration="2.774979589s" totalImagesPullingTime="1.412432443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:29:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:29:39.773572256 +0000 UTC m=+590.881918196" watchObservedRunningTime="2026-07-08 03:29:39.774979589 +0000 UTC m=+590.883325571"
Jul 08 03:29:39.790430 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:39.790388    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" podStartSLOduration=1.802458697 podStartE2EDuration="3.790373188s" totalImagesPullingTime="1.987914491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:29:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:29:39.788970813 +0000 UTC m=+590.897316776" watchObservedRunningTime="2026-07-08 03:29:39.790373188 +0000 UTC m=+590.898719128"
Jul 08 03:29:40.008489 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.008428    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:40.148385 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.148351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swd6r\" (UniqueName: \"kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r\") pod \"7e06825f-536f-4150-9c16-6137c7394214\" (UID: \"7e06825f-536f-4150-9c16-6137c7394214\") "
Jul 08 03:29:40.150682 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.150649    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r" (OuterVolumeSpecName: "kube-api-access-swd6r") pod "7e06825f-536f-4150-9c16-6137c7394214" (UID: "7e06825f-536f-4150-9c16-6137c7394214"). InnerVolumeSpecName "kube-api-access-swd6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:29:40.249992 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.249959    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swd6r\" (UniqueName: \"kubernetes.io/projected/7e06825f-536f-4150-9c16-6137c7394214-kube-api-access-swd6r\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:40.766088 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.766053    2563 generic.go:356] "Generic (PLEG): container finished" podID="7e06825f-536f-4150-9c16-6137c7394214" containerID="afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547" exitCode=0
Jul 08 03:29:40.766530 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.766107    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7"
Jul 08 03:29:40.766530 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.766124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" event={"ID":"7e06825f-536f-4150-9c16-6137c7394214","Type":"ContainerDied","Data":"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"}
Jul 08 03:29:40.766530 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.766192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-bbmj7" event={"ID":"7e06825f-536f-4150-9c16-6137c7394214","Type":"ContainerDied","Data":"e96c9329ab9d2a9c2e181e8521d75f66c5504fcf78dc4cf530a18ea7840caac5"}
Jul 08 03:29:40.766530 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.766211    2563 scope.go:117] "RemoveContainer" containerID="afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"
Jul 08 03:29:40.775638 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.775622    2563 scope.go:117] "RemoveContainer" containerID="afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"
Jul 08 03:29:40.775872 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:29:40.775852    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547\": container with ID starting with afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547 not found: ID does not exist" containerID="afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"
Jul 08 03:29:40.775916 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.775883    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547"} err="failed to get container status \"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547\": rpc error: code = NotFound desc = could not find container \"afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547\": container with ID starting with afcaa966a39713653889192482e34bd06f69c0d04e66bb2b5e29d58830fdb547 not found: ID does not exist"
Jul 08 03:29:40.787001 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.786980    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:40.790907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:40.790889    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-bbmj7"]
Jul 08 03:29:41.384741 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:41.384707    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e06825f-536f-4150-9c16-6137c7394214" path="/var/lib/kubelet/pods/7e06825f-536f-4150-9c16-6137c7394214/volumes"
Jul 08 03:29:47.436865 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:47.436828    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:47.437307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:47.436899    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:47.459355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:47.459328    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:47.821751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:47.821723    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2mnhl"
Jul 08 03:29:49.924529 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:49.924429    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:29:49.946684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:49.925203    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:29:49.946684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:49.928677    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:29:49.946684 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:49.929512    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:29:52.355792 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.355758    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"]
Jul 08 03:29:52.356182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.356091    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e06825f-536f-4150-9c16-6137c7394214" containerName="registry-server"
Jul 08 03:29:52.356182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.356102    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e06825f-536f-4150-9c16-6137c7394214" containerName="registry-server"
Jul 08 03:29:52.356261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.356186    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e06825f-536f-4150-9c16-6137c7394214" containerName="registry-server"
Jul 08 03:29:52.359599 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.359579    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.362076 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.362059    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7mzxj\""
Jul 08 03:29:52.370226 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.370196    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"]
Jul 08 03:29:52.459055 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.459014    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.459315 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.459068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.459315 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.459190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4b89\" (UniqueName: \"kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.560043 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.560006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.560317 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.560055    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.560317 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.560114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4b89\" (UniqueName: \"kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.560501 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.560478    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.560547 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.560499    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.570156 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.570119    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4b89\" (UniqueName: \"kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.668695 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.668600    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:52.803485 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.803453    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"]
Jul 08 03:29:52.805191 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:52.805163    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3249e46_b90b_4afd_be17_2718d25b1a7f.slice/crio-54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd WatchSource:0}: Error finding container 54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd: Status 404 returned error can't find the container with id 54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd
Jul 08 03:29:52.826352 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:52.826323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp" event={"ID":"f3249e46-b90b-4afd-be17-2718d25b1a7f","Type":"ContainerStarted","Data":"54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd"}
Jul 08 03:29:53.157456 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.157418    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"]
Jul 08 03:29:53.160989 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.160970    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.169095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.169071    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"]
Jul 08 03:29:53.265860 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.265825    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh78f\" (UniqueName: \"kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.266026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.265887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.266026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.265957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.367448 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.367415    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh78f\" (UniqueName: \"kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.367790 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.367470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.367790 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.367495    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.367863 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.367806    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.367897 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.367872    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.377831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.377807    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh78f\" (UniqueName: \"kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.470431 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.470348    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:53.560469 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.560424    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"]
Jul 08 03:29:53.565616 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.565587    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.571955 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.571929    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"]
Jul 08 03:29:53.613831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.613807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"]
Jul 08 03:29:53.616030 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:53.616002    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960f06b0_4537_4872_9133_f79d69b51320.slice/crio-e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392 WatchSource:0}: Error finding container e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392: Status 404 returned error can't find the container with id e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392
Jul 08 03:29:53.669585 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.669564    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.669673 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.669614    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cg8ql\" (UniqueName: \"kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.669715 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.669674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.770639 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.770546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.770639 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.770611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cg8ql\" (UniqueName: \"kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.770880 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.770675    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.770967 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.770907    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.771033 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.770988    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.779460 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.779436    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cg8ql\" (UniqueName: \"kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.833635 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.833603    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerID="8ab2efd248c5a005f96ac3496b04c18ea963623d30f5cd31374e1f86c822b893" exitCode=0
Jul 08 03:29:53.833814 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.833696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp" event={"ID":"f3249e46-b90b-4afd-be17-2718d25b1a7f","Type":"ContainerDied","Data":"8ab2efd248c5a005f96ac3496b04c18ea963623d30f5cd31374e1f86c822b893"}
Jul 08 03:29:53.835266 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.835247    2563 generic.go:356] "Generic (PLEG): container finished" podID="960f06b0-4537-4872-9133-f79d69b51320" containerID="b477a8558f4c2dc9ca80422a1fe887e91e2ced3499822b1b2e80d8131a580226" exitCode=0
Jul 08 03:29:53.835359 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.835333    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn" event={"ID":"960f06b0-4537-4872-9133-f79d69b51320","Type":"ContainerDied","Data":"b477a8558f4c2dc9ca80422a1fe887e91e2ced3499822b1b2e80d8131a580226"}
Jul 08 03:29:53.835402 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.835371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn" event={"ID":"960f06b0-4537-4872-9133-f79d69b51320","Type":"ContainerStarted","Data":"e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392"}
Jul 08 03:29:53.891026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.890998    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:53.963479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.963437    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"]
Jul 08 03:29:53.968108 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.968078    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:53.975427 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:53.975364    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"]
Jul 08 03:29:54.032129 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.032106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"]
Jul 08 03:29:54.033824 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:54.033800    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb94ee3b1_440d_4b24_bcfd_66b413ce3ce1.slice/crio-d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794 WatchSource:0}: Error finding container d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794: Status 404 returned error can't find the container with id d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794
Jul 08 03:29:54.072245 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.072216    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.072361 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.072276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjzb4\" (UniqueName: \"kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.072448 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.072426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.173266 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.173233    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.173440 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.173276    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.173440 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.173321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjzb4\" (UniqueName: \"kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.173660 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.173638    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.173762 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.173649    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.181950 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.181922    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjzb4\" (UniqueName: \"kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.280342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.280303    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:54.413190 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.413166    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"]
Jul 08 03:29:54.415290 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:29:54.415264    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8039744b_ec8b_4c9e_854b_b62b5ef5134d.slice/crio-3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687 WatchSource:0}: Error finding container 3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687: Status 404 returned error can't find the container with id 3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687
Jul 08 03:29:54.841832 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.841804    2563 generic.go:356] "Generic (PLEG): container finished" podID="960f06b0-4537-4872-9133-f79d69b51320" containerID="26553117e65e16187b34be4d7c9fd25360a0e63ff29d9fcef41e055b44de369d" exitCode=0
Jul 08 03:29:54.841942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.841883    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn" event={"ID":"960f06b0-4537-4872-9133-f79d69b51320","Type":"ContainerDied","Data":"26553117e65e16187b34be4d7c9fd25360a0e63ff29d9fcef41e055b44de369d"}
Jul 08 03:29:54.843458 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.843431    2563 generic.go:356] "Generic (PLEG): container finished" podID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerID="147c9041f0222a5c0c02385dc16f19fda532164ff09b4a373082430d05c02a32" exitCode=0
Jul 08 03:29:54.843524 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.843504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth" event={"ID":"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1","Type":"ContainerDied","Data":"147c9041f0222a5c0c02385dc16f19fda532164ff09b4a373082430d05c02a32"}
Jul 08 03:29:54.843585 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.843545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth" event={"ID":"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1","Type":"ContainerStarted","Data":"d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794"}
Jul 08 03:29:54.845098 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.845074    2563 generic.go:356] "Generic (PLEG): container finished" podID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerID="0dfd3804b858f44c6fcc877b6cb24647b09cb452f505f32e1559e896d8315181" exitCode=0
Jul 08 03:29:54.845220 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.845128    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj" event={"ID":"8039744b-ec8b-4c9e-854b-b62b5ef5134d","Type":"ContainerDied","Data":"0dfd3804b858f44c6fcc877b6cb24647b09cb452f505f32e1559e896d8315181"}
Jul 08 03:29:54.845220 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:54.845179    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj" event={"ID":"8039744b-ec8b-4c9e-854b-b62b5ef5134d","Type":"ContainerStarted","Data":"3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687"}
Jul 08 03:29:55.854177 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:55.854149    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerID="9d5d19543fd68e03fba17a0297ad0e3d3f3511c4f8dbf2ca830613d0002c0c9a" exitCode=0
Jul 08 03:29:55.854523 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:55.854228    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp" event={"ID":"f3249e46-b90b-4afd-be17-2718d25b1a7f","Type":"ContainerDied","Data":"9d5d19543fd68e03fba17a0297ad0e3d3f3511c4f8dbf2ca830613d0002c0c9a"}
Jul 08 03:29:55.856568 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:55.856544    2563 generic.go:356] "Generic (PLEG): container finished" podID="960f06b0-4537-4872-9133-f79d69b51320" containerID="6085ce5ed45d64ea3c90add51df46adaf2c6b19afb1d01fd2e9eb596fcf05440" exitCode=0
Jul 08 03:29:55.856687 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:55.856622    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn" event={"ID":"960f06b0-4537-4872-9133-f79d69b51320","Type":"ContainerDied","Data":"6085ce5ed45d64ea3c90add51df46adaf2c6b19afb1d01fd2e9eb596fcf05440"}
Jul 08 03:29:56.863853 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.863772    2563 generic.go:356] "Generic (PLEG): container finished" podID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerID="4c8348bbf6319421c54351c11680ffd82874abab7dd09c0f0547957b80d646cd" exitCode=0
Jul 08 03:29:56.864269 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.863861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth" event={"ID":"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1","Type":"ContainerDied","Data":"4c8348bbf6319421c54351c11680ffd82874abab7dd09c0f0547957b80d646cd"}
Jul 08 03:29:56.865782 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.865759    2563 generic.go:356] "Generic (PLEG): container finished" podID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerID="8f593bc47d45e0cd9b1d1d730a8b25b202f149e78200d0e0dcc8a6b266afa04e" exitCode=0
Jul 08 03:29:56.865912 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.865889    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj" event={"ID":"8039744b-ec8b-4c9e-854b-b62b5ef5134d","Type":"ContainerDied","Data":"8f593bc47d45e0cd9b1d1d730a8b25b202f149e78200d0e0dcc8a6b266afa04e"}
Jul 08 03:29:56.868178 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.868154    2563 generic.go:356] "Generic (PLEG): container finished" podID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerID="98ff31db963cff0e6a104383ef57f3cb22a81139d7bb1d666a019cef334d1539" exitCode=0
Jul 08 03:29:56.868256 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:56.868224    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp" event={"ID":"f3249e46-b90b-4afd-be17-2718d25b1a7f","Type":"ContainerDied","Data":"98ff31db963cff0e6a104383ef57f3cb22a81139d7bb1d666a019cef334d1539"}
Jul 08 03:29:57.015512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.015487    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:57.098507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.098476    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle\") pod \"960f06b0-4537-4872-9133-f79d69b51320\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") "
Jul 08 03:29:57.098679 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.098519    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh78f\" (UniqueName: \"kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f\") pod \"960f06b0-4537-4872-9133-f79d69b51320\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") "
Jul 08 03:29:57.098679 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.098535    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util\") pod \"960f06b0-4537-4872-9133-f79d69b51320\" (UID: \"960f06b0-4537-4872-9133-f79d69b51320\") "
Jul 08 03:29:57.099113 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.099090    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle" (OuterVolumeSpecName: "bundle") pod "960f06b0-4537-4872-9133-f79d69b51320" (UID: "960f06b0-4537-4872-9133-f79d69b51320"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:57.100626 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.100606    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f" (OuterVolumeSpecName: "kube-api-access-zh78f") pod "960f06b0-4537-4872-9133-f79d69b51320" (UID: "960f06b0-4537-4872-9133-f79d69b51320"). InnerVolumeSpecName "kube-api-access-zh78f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:29:57.104075 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.104052    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util" (OuterVolumeSpecName: "util") pod "960f06b0-4537-4872-9133-f79d69b51320" (UID: "960f06b0-4537-4872-9133-f79d69b51320"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:57.199424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.199345    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:57.199424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.199381    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh78f\" (UniqueName: \"kubernetes.io/projected/960f06b0-4537-4872-9133-f79d69b51320-kube-api-access-zh78f\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:57.199424 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.199398    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/960f06b0-4537-4872-9133-f79d69b51320-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:57.879293 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.879205    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn"
Jul 08 03:29:57.879752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.879207    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn" event={"ID":"960f06b0-4537-4872-9133-f79d69b51320","Type":"ContainerDied","Data":"e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392"}
Jul 08 03:29:57.879752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.879368    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e3f8cdb3ec1172570fab93b4d665b2ad00d5e8c5eb671e0695dd2ee6def0f392"
Jul 08 03:29:57.881483 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.881452    2563 generic.go:356] "Generic (PLEG): container finished" podID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerID="695c11998ed80d854b876c31a1efb0c956eb0f59c2dab381eccb9eed5cead3b3" exitCode=0
Jul 08 03:29:57.881612 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.881560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth" event={"ID":"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1","Type":"ContainerDied","Data":"695c11998ed80d854b876c31a1efb0c956eb0f59c2dab381eccb9eed5cead3b3"}
Jul 08 03:29:57.883677 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.883656    2563 generic.go:356] "Generic (PLEG): container finished" podID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerID="482f10a5a413d1bd77517ca9c8436c81c73f47ad0df42dcf0298bcb034f7fdb9" exitCode=0
Jul 08 03:29:57.883784 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:57.883685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj" event={"ID":"8039744b-ec8b-4c9e-854b-b62b5ef5134d","Type":"ContainerDied","Data":"482f10a5a413d1bd77517ca9c8436c81c73f47ad0df42dcf0298bcb034f7fdb9"}
Jul 08 03:29:58.015538 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.015514    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:58.108394 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.108364    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util\") pod \"f3249e46-b90b-4afd-be17-2718d25b1a7f\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") "
Jul 08 03:29:58.108600 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.108409    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4b89\" (UniqueName: \"kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89\") pod \"f3249e46-b90b-4afd-be17-2718d25b1a7f\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") "
Jul 08 03:29:58.108600 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.108490    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle\") pod \"f3249e46-b90b-4afd-be17-2718d25b1a7f\" (UID: \"f3249e46-b90b-4afd-be17-2718d25b1a7f\") "
Jul 08 03:29:58.108911 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.108887    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle" (OuterVolumeSpecName: "bundle") pod "f3249e46-b90b-4afd-be17-2718d25b1a7f" (UID: "f3249e46-b90b-4afd-be17-2718d25b1a7f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:58.110824 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.110804    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89" (OuterVolumeSpecName: "kube-api-access-p4b89") pod "f3249e46-b90b-4afd-be17-2718d25b1a7f" (UID: "f3249e46-b90b-4afd-be17-2718d25b1a7f"). InnerVolumeSpecName "kube-api-access-p4b89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:29:58.204165 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.204059    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util" (OuterVolumeSpecName: "util") pod "f3249e46-b90b-4afd-be17-2718d25b1a7f" (UID: "f3249e46-b90b-4afd-be17-2718d25b1a7f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:58.209284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.209264    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:58.209353 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.209289    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4b89\" (UniqueName: \"kubernetes.io/projected/f3249e46-b90b-4afd-be17-2718d25b1a7f-kube-api-access-p4b89\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:58.209353 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.209302    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f3249e46-b90b-4afd-be17-2718d25b1a7f-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:58.890928 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.890893    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp"
Jul 08 03:29:58.891340 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.890936    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp" event={"ID":"f3249e46-b90b-4afd-be17-2718d25b1a7f","Type":"ContainerDied","Data":"54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd"}
Jul 08 03:29:58.891340 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:58.890989    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54fbcc9a3deb250a78843aebbdb9a85a56945a625f17eb56f0828ce49d8621bd"
Jul 08 03:29:59.032838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.032817    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:59.045855 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.045835    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:29:59.118132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.118103    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cg8ql\" (UniqueName: \"kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql\") pod \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") "
Jul 08 03:29:59.118132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.118151    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle\") pod \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") "
Jul 08 03:29:59.118389 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.118249    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util\") pod \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\" (UID: \"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1\") "
Jul 08 03:29:59.118660 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.118637    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle" (OuterVolumeSpecName: "bundle") pod "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" (UID: "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:59.120397 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.120374    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql" (OuterVolumeSpecName: "kube-api-access-cg8ql") pod "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" (UID: "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1"). InnerVolumeSpecName "kube-api-access-cg8ql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:29:59.219454 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.219358    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjzb4\" (UniqueName: \"kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4\") pod \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") "
Jul 08 03:29:59.219454 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.219423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util\") pod \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") "
Jul 08 03:29:59.219670 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.219482    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle\") pod \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\" (UID: \"8039744b-ec8b-4c9e-854b-b62b5ef5134d\") "
Jul 08 03:29:59.219738 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.219717    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cg8ql\" (UniqueName: \"kubernetes.io/projected/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-kube-api-access-cg8ql\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.219788 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.219741    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.220097 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.220066    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle" (OuterVolumeSpecName: "bundle") pod "8039744b-ec8b-4c9e-854b-b62b5ef5134d" (UID: "8039744b-ec8b-4c9e-854b-b62b5ef5134d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:59.221703 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.221681    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4" (OuterVolumeSpecName: "kube-api-access-cjzb4") pod "8039744b-ec8b-4c9e-854b-b62b5ef5134d" (UID: "8039744b-ec8b-4c9e-854b-b62b5ef5134d"). InnerVolumeSpecName "kube-api-access-cjzb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:29:59.224773 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.224747    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util" (OuterVolumeSpecName: "util") pod "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" (UID: "b94ee3b1-440d-4b24-bcfd-66b413ce3ce1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:59.225394 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.225375    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util" (OuterVolumeSpecName: "util") pod "8039744b-ec8b-4c9e-854b-b62b5ef5134d" (UID: "8039744b-ec8b-4c9e-854b-b62b5ef5134d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:29:59.321013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.320976    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjzb4\" (UniqueName: \"kubernetes.io/projected/8039744b-ec8b-4c9e-854b-b62b5ef5134d-kube-api-access-cjzb4\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.321013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.321010    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.321235 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.321023    2563 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b94ee3b1-440d-4b24-bcfd-66b413ce3ce1-util\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.321235 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.321035    2563 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8039744b-ec8b-4c9e-854b-b62b5ef5134d-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:29:59.897728 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.897704    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth"
Jul 08 03:29:59.897728 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.897714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth" event={"ID":"b94ee3b1-440d-4b24-bcfd-66b413ce3ce1","Type":"ContainerDied","Data":"d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794"}
Jul 08 03:29:59.898229 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.897763    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d35284d8eaa768f7c8c89d4c14fa4653543a07410831ed08722aea87b26aa794"
Jul 08 03:29:59.899722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.899679    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj" event={"ID":"8039744b-ec8b-4c9e-854b-b62b5ef5134d","Type":"ContainerDied","Data":"3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687"}
Jul 08 03:29:59.899722 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.899718    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bf02f2f076970d25bb8f9a85f4e101b819aa030e65ab6fe91f71c3cf6a6d687"
Jul 08 03:29:59.899927 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:29:59.899723    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj"
Jul 08 03:30:09.516546 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.516507    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"]
Jul 08 03:30:09.517046 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517027    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517049    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517064    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517072    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517082    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="pull"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517090    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="pull"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517099    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517107    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517116    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517124    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="util"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517155    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="extract"
Jul 08 03:30:09.517175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517164    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517186    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517193    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517202    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517210    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517221    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517228    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517240    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517249    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517261    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517268    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="pull"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517282    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517290    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517389    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="960f06b0-4537-4872-9133-f79d69b51320" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517403    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="8039744b-ec8b-4c9e-854b-b62b5ef5134d" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517412    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94ee3b1-440d-4b24-bcfd-66b413ce3ce1" containerName="extract"
Jul 08 03:30:09.517669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.517420    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3249e46-b90b-4afd-be17-2718d25b1a7f" containerName="extract"
Jul 08 03:30:09.525851 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.525825    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:09.529176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.529157    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kxwr8\""
Jul 08 03:30:09.532823 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.532795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"]
Jul 08 03:30:09.603692 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.603608    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbgnm\" (UniqueName: \"kubernetes.io/projected/d62ec88a-5e14-4243-836b-8b0e6f70daff-kube-api-access-wbgnm\") pod \"authorino-operator-75cc8d5cb7-z9p6p\" (UID: \"d62ec88a-5e14-4243-836b-8b0e6f70daff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:09.704681 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.704645    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbgnm\" (UniqueName: \"kubernetes.io/projected/d62ec88a-5e14-4243-836b-8b0e6f70daff-kube-api-access-wbgnm\") pod \"authorino-operator-75cc8d5cb7-z9p6p\" (UID: \"d62ec88a-5e14-4243-836b-8b0e6f70daff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:09.715732 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.715701    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbgnm\" (UniqueName: \"kubernetes.io/projected/d62ec88a-5e14-4243-836b-8b0e6f70daff-kube-api-access-wbgnm\") pod \"authorino-operator-75cc8d5cb7-z9p6p\" (UID: \"d62ec88a-5e14-4243-836b-8b0e6f70daff\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:09.837704 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.837670    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:09.982790 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.982757    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"]
Jul 08 03:30:09.985056 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:09.985032    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62ec88a_5e14_4243_836b_8b0e6f70daff.slice/crio-3c53bd2c9d84cf4f5e3649687c92716c1f1ee92bc59aadcfefbfb9ef9a77d3f6 WatchSource:0}: Error finding container 3c53bd2c9d84cf4f5e3649687c92716c1f1ee92bc59aadcfefbfb9ef9a77d3f6: Status 404 returned error can't find the container with id 3c53bd2c9d84cf4f5e3649687c92716c1f1ee92bc59aadcfefbfb9ef9a77d3f6
Jul 08 03:30:09.991358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:09.991336    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:30:10.959426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:10.959382    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p" event={"ID":"d62ec88a-5e14-4243-836b-8b0e6f70daff","Type":"ContainerStarted","Data":"3c53bd2c9d84cf4f5e3649687c92716c1f1ee92bc59aadcfefbfb9ef9a77d3f6"}
Jul 08 03:30:12.505884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.505855    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"]
Jul 08 03:30:12.509392 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.509374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:12.511665 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.511647    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 08 03:30:12.511776 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.511758    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-lzx4h\""
Jul 08 03:30:12.521333 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.521308    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"]
Jul 08 03:30:12.531764 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.531742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqhpg\" (UniqueName: \"kubernetes.io/projected/fdf922e1-1caa-41c6-9f2a-2817899411a0-kube-api-access-sqhpg\") pod \"dns-operator-controller-manager-864dd78c4c-95jls\" (UID: \"fdf922e1-1caa-41c6-9f2a-2817899411a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:12.632642 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.632599    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqhpg\" (UniqueName: \"kubernetes.io/projected/fdf922e1-1caa-41c6-9f2a-2817899411a0-kube-api-access-sqhpg\") pod \"dns-operator-controller-manager-864dd78c4c-95jls\" (UID: \"fdf922e1-1caa-41c6-9f2a-2817899411a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:12.645371 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.645343    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqhpg\" (UniqueName: \"kubernetes.io/projected/fdf922e1-1caa-41c6-9f2a-2817899411a0-kube-api-access-sqhpg\") pod \"dns-operator-controller-manager-864dd78c4c-95jls\" (UID: \"fdf922e1-1caa-41c6-9f2a-2817899411a0\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:12.818988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.818938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:12.953347 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.953323    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"]
Jul 08 03:30:12.955506 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:12.955479    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf922e1_1caa_41c6_9f2a_2817899411a0.slice/crio-d73cbad7132e76c48ae53a7484e7a60ce8c10b5232b8fe8cb17dbfb651a1d06d WatchSource:0}: Error finding container d73cbad7132e76c48ae53a7484e7a60ce8c10b5232b8fe8cb17dbfb651a1d06d: Status 404 returned error can't find the container with id d73cbad7132e76c48ae53a7484e7a60ce8c10b5232b8fe8cb17dbfb651a1d06d
Jul 08 03:30:12.973061 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.973032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p" event={"ID":"d62ec88a-5e14-4243-836b-8b0e6f70daff","Type":"ContainerStarted","Data":"e9f2d9f1cf35e7cef4bc1dd2abef3cb3708f9f0ef02279aa42898bfe88cd9658"}
Jul 08 03:30:12.973218 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.973171    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:12.974281 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.974254    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls" event={"ID":"fdf922e1-1caa-41c6-9f2a-2817899411a0","Type":"ContainerStarted","Data":"d73cbad7132e76c48ae53a7484e7a60ce8c10b5232b8fe8cb17dbfb651a1d06d"}
Jul 08 03:30:12.997516 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:12.997476    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p" podStartSLOduration=1.554413515 podStartE2EDuration="3.997461223s" totalImagesPullingTime="2.443047708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:12.996001369 +0000 UTC m=+624.104347310" watchObservedRunningTime="2026-07-08 03:30:12.997461223 +0000 UTC m=+624.105807227"
Jul 08 03:30:15.852188 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.852153    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"]
Jul 08 03:30:15.855637 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.855620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:15.858319 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.858297    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-q4zsn\""
Jul 08 03:30:15.867126 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.867101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"]
Jul 08 03:30:15.964080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.964051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bhtj\" (UniqueName: \"kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj\") pod \"limitador-operator-controller-manager-859d7b78bd-726xm\" (UID: \"21b0c045-e0e7-4127-bd7f-63c5aba5ae93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:15.997809 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.997773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls" event={"ID":"fdf922e1-1caa-41c6-9f2a-2817899411a0","Type":"ContainerStarted","Data":"bf86895a43fb669e0aaf0810fb4032d380ce4c8cadcc466b68e680b475f49f15"}
Jul 08 03:30:15.997963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:15.997832    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:16.019982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.019933    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls" podStartSLOduration=1.831576551 podStartE2EDuration="4.019918114s" totalImagesPullingTime="2.188341563s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:16.018011522 +0000 UTC m=+627.126357467" watchObservedRunningTime="2026-07-08 03:30:16.019918114 +0000 UTC m=+627.128264053"
Jul 08 03:30:16.064875 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.064844    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bhtj\" (UniqueName: \"kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj\") pod \"limitador-operator-controller-manager-859d7b78bd-726xm\" (UID: \"21b0c045-e0e7-4127-bd7f-63c5aba5ae93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:16.079315 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.079284    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bhtj\" (UniqueName: \"kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj\") pod \"limitador-operator-controller-manager-859d7b78bd-726xm\" (UID: \"21b0c045-e0e7-4127-bd7f-63c5aba5ae93\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:16.166128 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.166041    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:16.329013 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.328984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"]
Jul 08 03:30:16.331552 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:16.331526    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21b0c045_e0e7_4127_bd7f_63c5aba5ae93.slice/crio-54412429b889ca41aaf657647b0b62fecf8190e10e15aa5eebcc89c1b85b81f6 WatchSource:0}: Error finding container 54412429b889ca41aaf657647b0b62fecf8190e10e15aa5eebcc89c1b85b81f6: Status 404 returned error can't find the container with id 54412429b889ca41aaf657647b0b62fecf8190e10e15aa5eebcc89c1b85b81f6
Jul 08 03:30:16.527775 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.527692    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7874684746-58smt"]
Jul 08 03:30:16.533020 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.532994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.546850 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.546825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7874684746-58smt"]
Jul 08 03:30:16.569243 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569215    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-trusted-ca-bundle\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-oauth-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569299    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-service-ca\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569354    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-oauth-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lltgv\" (UniqueName: \"kubernetes.io/projected/4f024b8b-29db-48fd-93a7-6681184253e0-kube-api-access-lltgv\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-console-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.569683 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.569494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670321 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670283    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-console-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-trusted-ca-bundle\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670621    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-oauth-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670700    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-service-ca\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-oauth-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.670769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.670760    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lltgv\" (UniqueName: \"kubernetes.io/projected/4f024b8b-29db-48fd-93a7-6681184253e0-kube-api-access-lltgv\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.671235 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.671208    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-console-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.671440 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.671384    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-oauth-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.671595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.671572    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-service-ca\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.671792 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.671775    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4f024b8b-29db-48fd-93a7-6681184253e0-trusted-ca-bundle\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.673176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.673120    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-serving-cert\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.673176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.673122    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4f024b8b-29db-48fd-93a7-6681184253e0-console-oauth-config\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.689209 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.689184    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lltgv\" (UniqueName: \"kubernetes.io/projected/4f024b8b-29db-48fd-93a7-6681184253e0-kube-api-access-lltgv\") pod \"console-7874684746-58smt\" (UID: \"4f024b8b-29db-48fd-93a7-6681184253e0\") " pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:16.843432 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:16.843399    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:17.011332 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:17.011126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm" event={"ID":"21b0c045-e0e7-4127-bd7f-63c5aba5ae93","Type":"ContainerStarted","Data":"54412429b889ca41aaf657647b0b62fecf8190e10e15aa5eebcc89c1b85b81f6"}
Jul 08 03:30:17.123269 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:17.123242    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7874684746-58smt"]
Jul 08 03:30:17.171649 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:17.171618    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f024b8b_29db_48fd_93a7_6681184253e0.slice/crio-a9aa5d95992014362b4006d3655e08dd66b7bcda02887e3409d5a56f88a0d121 WatchSource:0}: Error finding container a9aa5d95992014362b4006d3655e08dd66b7bcda02887e3409d5a56f88a0d121: Status 404 returned error can't find the container with id a9aa5d95992014362b4006d3655e08dd66b7bcda02887e3409d5a56f88a0d121
Jul 08 03:30:18.020925 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:18.020892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7874684746-58smt" event={"ID":"4f024b8b-29db-48fd-93a7-6681184253e0","Type":"ContainerStarted","Data":"1b22a76aaaa8ac32db66e2a9d6133c2d0df11d3aca109543fa6efdd38fd77b70"}
Jul 08 03:30:18.021324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:18.020954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7874684746-58smt" event={"ID":"4f024b8b-29db-48fd-93a7-6681184253e0","Type":"ContainerStarted","Data":"a9aa5d95992014362b4006d3655e08dd66b7bcda02887e3409d5a56f88a0d121"}
Jul 08 03:30:18.044495 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:18.044441    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7874684746-58smt" podStartSLOduration=2.0444217670000002 podStartE2EDuration="2.044421767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:18.041865823 +0000 UTC m=+629.150211764" watchObservedRunningTime="2026-07-08 03:30:18.044421767 +0000 UTC m=+629.152767706"
Jul 08 03:30:19.033537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:19.033470    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm" event={"ID":"21b0c045-e0e7-4127-bd7f-63c5aba5ae93","Type":"ContainerStarted","Data":"ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac"}
Jul 08 03:30:19.034017 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:19.033550    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:19.062783 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:19.062717    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm" podStartSLOduration=2.43484878 podStartE2EDuration="4.062695257s" totalImagesPullingTime="1.627846477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:19.054025327 +0000 UTC m=+630.162371272" watchObservedRunningTime="2026-07-08 03:30:19.062695257 +0000 UTC m=+630.171041200"
Jul 08 03:30:20.087837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.087799    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"]
Jul 08 03:30:20.091432 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.091414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.093989 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.093968    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d2xbg\""
Jul 08 03:30:20.106305 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.106276    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"]
Jul 08 03:30:20.204514 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.204471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cb6h\" (UniqueName: \"kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.204681 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.204529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.305705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.305674    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cb6h\" (UniqueName: \"kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.305874 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.305743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.306124 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.306102    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.316475 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.316450    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cb6h\" (UniqueName: \"kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.401764 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.401676    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:20.547940 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:20.547915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"]
Jul 08 03:30:20.549742 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:20.549714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432b42a9_7fe1_4b2d_ab7b_0058d0c3fdb8.slice/crio-508a1e341a7a6e52542d6e4ee2630cdfb83fb2930e8541d75e4b57fa6ff43824 WatchSource:0}: Error finding container 508a1e341a7a6e52542d6e4ee2630cdfb83fb2930e8541d75e4b57fa6ff43824: Status 404 returned error can't find the container with id 508a1e341a7a6e52542d6e4ee2630cdfb83fb2930e8541d75e4b57fa6ff43824
Jul 08 03:30:21.044639 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:21.044599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6" event={"ID":"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8","Type":"ContainerStarted","Data":"508a1e341a7a6e52542d6e4ee2630cdfb83fb2930e8541d75e4b57fa6ff43824"}
Jul 08 03:30:23.982378 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:23.982347    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-z9p6p"
Jul 08 03:30:26.844360 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:26.844317    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:26.844850 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:26.844473    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:26.849995 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:26.849971    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:27.014397 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:27.014366    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-95jls"
Jul 08 03:30:27.090858 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:27.090830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7874684746-58smt"
Jul 08 03:30:27.141516 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:27.141438    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:30:29.098392 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:29.098349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6" event={"ID":"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8","Type":"ContainerStarted","Data":"926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134"}
Jul 08 03:30:29.141341 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:29.141296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6" podStartSLOduration=1.497795756 podStartE2EDuration="9.141265763s" totalImagesPullingTime="7.643470007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:29.14066081 +0000 UTC m=+640.249006768" watchObservedRunningTime="2026-07-08 03:30:29.141265763 +0000 UTC m=+640.249611700"
Jul 08 03:30:30.103916 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:30.103888    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:31.047580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:31.047547    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:41.112883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:41.112852    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:42.439691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.439657    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"]
Jul 08 03:30:42.443008 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.442993    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.471848 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.471820    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"]
Jul 08 03:30:42.624983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.624955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.625218 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.625005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfcxp\" (UniqueName: \"kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.725348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.725263    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"]
Jul 08 03:30:42.725549 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.725531    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-lfcxp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f"
Jul 08 03:30:42.725595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.725570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfcxp\" (UniqueName: \"kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.725649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.725634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.725950 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.725936    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:42.729865 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.729844    2563 projected.go:196] Error preparing data for projected volume kube-api-access-lfcxp for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" not found
Jul 08 03:30:42.730019 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.729914    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp podName:bcf4deae-3fe7-4b28-b344-b2610a45a31f nodeName:}" failed. No retries permitted until 2026-07-08 03:30:43.229895286 +0000 UTC m=+654.338241209 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lfcxp" (UniqueName: "kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp") pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" (UID: "bcf4deae-3fe7-4b28-b344-b2610a45a31f") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" not found
Jul 08 03:30:42.730593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.730572    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"]
Jul 08 03:30:42.742187 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.742159    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"]
Jul 08 03:30:42.742334 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.742314    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" containerName="manager" containerID="cri-o://926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134" gracePeriod=2
Jul 08 03:30:42.755199 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.755169    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"]
Jul 08 03:30:42.756109 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.756069    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\": pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:42.772543 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.772519    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"]
Jul 08 03:30:42.772760 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.772737    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" containerName="manager" containerID="cri-o://ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac" gracePeriod=2
Jul 08 03:30:42.774556 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.774522    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:42.775412 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.775390    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:30:42.775826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.775805    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" containerName="manager"
Jul 08 03:30:42.775826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.775827    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" containerName="manager"
Jul 08 03:30:42.775963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.775956    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" containerName="manager"
Jul 08 03:30:42.779236 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.779203    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:42.780428 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.780188    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"]
Jul 08 03:30:42.794982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.794946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:30:42.807830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.807804    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"]
Jul 08 03:30:42.808243 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.808224    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" containerName="manager"
Jul 08 03:30:42.808355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.808245    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" containerName="manager"
Jul 08 03:30:42.808355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.808336    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" containerName="manager"
Jul 08 03:30:42.810872 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.810844    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:42.811767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.811750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:42.813465 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.813439    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:42.815105 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.815081    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:42.816920 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:42.816888    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:42.825561 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.825535    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"]
Jul 08 03:30:42.927583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.927532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52ffl\" (UniqueName: \"kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:42.927769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.927601    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt8b9\" (UniqueName: \"kubernetes.io/projected/cb870b59-f919-45ce-9977-38d8ca77dfaf-kube-api-access-kt8b9\") pod \"limitador-operator-controller-manager-859d7b78bd-ztm2h\" (UID: \"cb870b59-f919-45ce-9977-38d8ca77dfaf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:42.927769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:42.927632    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.019828 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.019806    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.021873 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.021850    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.022934 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.022911    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.023542 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.023521    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.025039 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.025022    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.026757 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.026734    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.028171 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.028123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52ffl\" (UniqueName: \"kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.028251 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.028215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt8b9\" (UniqueName: \"kubernetes.io/projected/cb870b59-f919-45ce-9977-38d8ca77dfaf-kube-api-access-kt8b9\") pod \"limitador-operator-controller-manager-859d7b78bd-ztm2h\" (UID: \"cb870b59-f919-45ce-9977-38d8ca77dfaf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:43.028315 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.028247    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.028567 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.028547    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.036236 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.036215    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-52ffl\" (UniqueName: \"kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl\") pod \"kuadrant-operator-controller-manager-8684c9ff49-tr9nt\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.036318 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.036298    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt8b9\" (UniqueName: \"kubernetes.io/projected/cb870b59-f919-45ce-9977-38d8ca77dfaf-kube-api-access-kt8b9\") pod \"limitador-operator-controller-manager-859d7b78bd-ztm2h\" (UID: \"cb870b59-f919-45ce-9977-38d8ca77dfaf\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:43.128891 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.128860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cb6h\" (UniqueName: \"kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h\") pod \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") "
Jul 08 03:30:43.129045 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.128923    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume\") pod \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\" (UID: \"432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8\") "
Jul 08 03:30:43.129132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.129102    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bhtj\" (UniqueName: \"kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj\") pod \"21b0c045-e0e7-4127-bd7f-63c5aba5ae93\" (UID: \"21b0c045-e0e7-4127-bd7f-63c5aba5ae93\") "
Jul 08 03:30:43.129398 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.129376    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" (UID: "432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:30:43.131063 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.131043    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h" (OuterVolumeSpecName: "kube-api-access-2cb6h") pod "432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" (UID: "432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8"). InnerVolumeSpecName "kube-api-access-2cb6h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:30:43.131118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.131100    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj" (OuterVolumeSpecName: "kube-api-access-5bhtj") pod "21b0c045-e0e7-4127-bd7f-63c5aba5ae93" (UID: "21b0c045-e0e7-4127-bd7f-63c5aba5ae93"). InnerVolumeSpecName "kube-api-access-5bhtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:30:43.182514 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.182485    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:43.186453 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.186423    2563 generic.go:356] "Generic (PLEG): container finished" podID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" containerID="ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac" exitCode=0
Jul 08 03:30:43.186525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.186489    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.186616 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.186572    2563 scope.go:117] "RemoveContainer" containerID="ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac"
Jul 08 03:30:43.187474 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.187450    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:43.188718 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.188692    2563 generic.go:356] "Generic (PLEG): container finished" podID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" containerID="926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134" exitCode=0
Jul 08 03:30:43.188827 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.188743    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.188895 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.188750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.189490 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.189463    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.191192 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.191108    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.192879 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.192860    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.194487 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.194458    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.196090 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.196066    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.198672 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.198653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.200104 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.200081    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.202342 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.201913    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.203662 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.203521    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.206506 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.206460    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-7dhf6\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7dhf6"
Jul 08 03:30:43.206597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.206574    2563 scope.go:117] "RemoveContainer" containerID="ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac"
Jul 08 03:30:43.206953 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.206895    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac\": container with ID starting with ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac not found: ID does not exist" containerID="ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac"
Jul 08 03:30:43.206953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.206935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac"} err="failed to get container status \"ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac\": rpc error: code = NotFound desc = could not find container \"ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac\": container with ID starting with ff9c1c3fc5918fc0f2afe7d80d4a16d088e645d4ad82caed8f8d3b528c59c8ac not found: ID does not exist"
Jul 08 03:30:43.206953 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.206952    2563 scope.go:117] "RemoveContainer" containerID="926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134"
Jul 08 03:30:43.208166 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.208126    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-726xm\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-726xm"
Jul 08 03:30:43.210414 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.210378    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.218264 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.218242    2563 scope.go:117] "RemoveContainer" containerID="926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134"
Jul 08 03:30:43.218644 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.218618    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134\": container with ID starting with 926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134 not found: ID does not exist" containerID="926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134"
Jul 08 03:30:43.218735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.218658    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134"} err="failed to get container status \"926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134\": rpc error: code = NotFound desc = could not find container \"926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134\": container with ID starting with 926cb9a8340396378cedfdacb02139c042b8ad96a2fd04fce9c7d559e316a134 not found: ID does not exist"
Jul 08 03:30:43.230183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.230123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfcxp\" (UniqueName: \"kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp\") pod \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:43.230360 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.230259    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cb6h\" (UniqueName: \"kubernetes.io/projected/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-kube-api-access-2cb6h\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:43.230360 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.230271    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8-extensions-socket-volume\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:43.230360 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.230280    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bhtj\" (UniqueName: \"kubernetes.io/projected/21b0c045-e0e7-4127-bd7f-63c5aba5ae93-kube-api-access-5bhtj\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:43.232867 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.232844    2563 projected.go:196] Error preparing data for projected volume kube-api-access-lfcxp for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" not found
Jul 08 03:30:43.233002 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:43.232913    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp podName:bcf4deae-3fe7-4b28-b344-b2610a45a31f nodeName:}" failed. No retries permitted until 2026-07-08 03:30:44.232896789 +0000 UTC m=+655.341242716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lfcxp" (UniqueName: "kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp") pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" (UID: "bcf4deae-3fe7-4b28-b344-b2610a45a31f") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-jvbl2" not found
Jul 08 03:30:43.331666 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.331635    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume\") pod \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\" (UID: \"bcf4deae-3fe7-4b28-b344-b2610a45a31f\") "
Jul 08 03:30:43.331850 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.331833    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfcxp\" (UniqueName: \"kubernetes.io/projected/bcf4deae-3fe7-4b28-b344-b2610a45a31f-kube-api-access-lfcxp\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:43.331966 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.331935    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bcf4deae-3fe7-4b28-b344-b2610a45a31f" (UID: "bcf4deae-3fe7-4b28-b344-b2610a45a31f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:30:43.345091 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.345065    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:30:43.347575 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:43.347545    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4465aefb_0724_4a5e_82ce_06c1569fef66.slice/crio-fffff13fd70e23c68f08cd6b1bd5d829c51259fd65ef6fbe048ce5fccd994b62 WatchSource:0}: Error finding container fffff13fd70e23c68f08cd6b1bd5d829c51259fd65ef6fbe048ce5fccd994b62: Status 404 returned error can't find the container with id fffff13fd70e23c68f08cd6b1bd5d829c51259fd65ef6fbe048ce5fccd994b62
Jul 08 03:30:43.371363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.371340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"]
Jul 08 03:30:43.372288 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:30:43.372262    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb870b59_f919_45ce_9977_38d8ca77dfaf.slice/crio-853eb179ed282beb720a215fec5f15235d68d494462b0d819dbd3cf92e6ba5ce WatchSource:0}: Error finding container 853eb179ed282beb720a215fec5f15235d68d494462b0d819dbd3cf92e6ba5ce: Status 404 returned error can't find the container with id 853eb179ed282beb720a215fec5f15235d68d494462b0d819dbd3cf92e6ba5ce
Jul 08 03:30:43.385594 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.385570    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21b0c045-e0e7-4127-bd7f-63c5aba5ae93" path="/var/lib/kubelet/pods/21b0c045-e0e7-4127-bd7f-63c5aba5ae93/volumes"
Jul 08 03:30:43.385909 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.385896    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8" path="/var/lib/kubelet/pods/432b42a9-7fe1-4b2d-ab7b-0058d0c3fdb8/volumes"
Jul 08 03:30:43.386210 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.386197    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f" path="/var/lib/kubelet/pods/bcf4deae-3fe7-4b28-b344-b2610a45a31f/volumes"
Jul 08 03:30:43.432368 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:43.432344    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bcf4deae-3fe7-4b28-b344-b2610a45a31f-extensions-socket-volume\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:44.198476 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.198431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h" event={"ID":"cb870b59-f919-45ce-9977-38d8ca77dfaf","Type":"ContainerStarted","Data":"aaf11f0365daebb3e7609aad9c79d58cf9a8999bf6cb7812572e4ec393a6ba2a"}
Jul 08 03:30:44.198963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.198493    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h" event={"ID":"cb870b59-f919-45ce-9977-38d8ca77dfaf","Type":"ContainerStarted","Data":"853eb179ed282beb720a215fec5f15235d68d494462b0d819dbd3cf92e6ba5ce"}
Jul 08 03:30:44.198963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.198594    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:30:44.200188 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.200169    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:44.200188 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.200174    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" event={"ID":"4465aefb-0724-4a5e-82ce-06c1569fef66","Type":"ContainerStarted","Data":"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"}
Jul 08 03:30:44.200389 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.200220    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" event={"ID":"4465aefb-0724-4a5e-82ce-06c1569fef66","Type":"ContainerStarted","Data":"fffff13fd70e23c68f08cd6b1bd5d829c51259fd65ef6fbe048ce5fccd994b62"}
Jul 08 03:30:44.200389 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.200264    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:44.242708 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.242657    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h" podStartSLOduration=2.24264005 podStartE2EDuration="2.24264005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:44.240985897 +0000 UTC m=+655.349331837" watchObservedRunningTime="2026-07-08 03:30:44.24264005 +0000 UTC m=+655.350986062"
Jul 08 03:30:44.274292 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:44.274242    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" podStartSLOduration=2.274216707 podStartE2EDuration="2.274216707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:30:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:30:44.272003962 +0000 UTC m=+655.380349949" watchObservedRunningTime="2026-07-08 03:30:44.274216707 +0000 UTC m=+655.382562648"
Jul 08 03:30:44.277621 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:44.277594    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-jvbl2\" is forbidden: User \"system:node:ip-10-0-137-160.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-160.ec2.internal' and this object" podUID="bcf4deae-3fe7-4b28-b344-b2610a45a31f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-jvbl2"
Jul 08 03:30:52.172719 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.172678    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-56fb6bb6f5-vrh5k" podUID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" containerName="console" containerID="cri-o://945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3" gracePeriod=15
Jul 08 03:30:52.427956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.427894    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:30:52.509217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509186    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7gz7v\" (UniqueName: \"kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509421 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509236    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509421 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509360    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509421 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509601 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509601 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509601 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509503    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle\") pod \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\" (UID: \"e172fd0d-9d45-4d41-af3e-e91fc6ee996c\") "
Jul 08 03:30:52.509752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509662    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca" (OuterVolumeSpecName: "service-ca") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:30:52.509883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509867    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.509988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509926    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:30:52.509988 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.509931    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config" (OuterVolumeSpecName: "console-config") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:30:52.510197 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.510026    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:30:52.511837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.511811    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:30:52.512068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.512045    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:30:52.512133 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.512080    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v" (OuterVolumeSpecName: "kube-api-access-7gz7v") pod "e172fd0d-9d45-4d41-af3e-e91fc6ee996c" (UID: "e172fd0d-9d45-4d41-af3e-e91fc6ee996c"). InnerVolumeSpecName "kube-api-access-7gz7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:30:52.611022 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.610986    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-trusted-ca-bundle\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.611022 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.611016    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7gz7v\" (UniqueName: \"kubernetes.io/projected/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-kube-api-access-7gz7v\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.611022 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.611026    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-oauth-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.611310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.611036    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.611310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.611047    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-oauth-serving-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:52.611310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:52.611056    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e172fd0d-9d45-4d41-af3e-e91fc6ee996c-console-config\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:30:53.251846 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.251808    2563 generic.go:356] "Generic (PLEG): container finished" podID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" containerID="945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3" exitCode=0
Jul 08 03:30:53.252359 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.251874    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56fb6bb6f5-vrh5k"
Jul 08 03:30:53.252359 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.251895    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fb6bb6f5-vrh5k" event={"ID":"e172fd0d-9d45-4d41-af3e-e91fc6ee996c","Type":"ContainerDied","Data":"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"}
Jul 08 03:30:53.252359 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.251977    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56fb6bb6f5-vrh5k" event={"ID":"e172fd0d-9d45-4d41-af3e-e91fc6ee996c","Type":"ContainerDied","Data":"f373d8cf77c70e9c4e043559c359e780a0ed0ade4a1c765d46cdbc2f609bb258"}
Jul 08 03:30:53.252359 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.251998    2563 scope.go:117] "RemoveContainer" containerID="945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"
Jul 08 03:30:53.261504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.261487    2563 scope.go:117] "RemoveContainer" containerID="945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"
Jul 08 03:30:53.261762 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:30:53.261745    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3\": container with ID starting with 945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3 not found: ID does not exist" containerID="945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"
Jul 08 03:30:53.261810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.261773    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3"} err="failed to get container status \"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3\": rpc error: code = NotFound desc = could not find container \"945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3\": container with ID starting with 945dd51fa6fbba1a956fa8ea936e947da42bc240d2402ab92d8f108d1ce37de3 not found: ID does not exist"
Jul 08 03:30:53.273957 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.273932    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:30:53.276059 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.276042    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56fb6bb6f5-vrh5k"]
Jul 08 03:30:53.385750 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:53.385715    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" path="/var/lib/kubelet/pods/e172fd0d-9d45-4d41-af3e-e91fc6ee996c/volumes"
Jul 08 03:30:55.208930 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:55.208898    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:30:55.209429 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:30:55.208979    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-ztm2h"
Jul 08 03:31:12.259253 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.259213    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:31:12.259650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.259434    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" podUID="4465aefb-0724-4a5e-82ce-06c1569fef66" containerName="manager" containerID="cri-o://954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455" gracePeriod=10
Jul 08 03:31:12.505173 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.505117    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:31:12.505531 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.505519    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" containerName="console"
Jul 08 03:31:12.505571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.505533    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" containerName="console"
Jul 08 03:31:12.505606 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.505601    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e172fd0d-9d45-4d41-af3e-e91fc6ee996c" containerName="console"
Jul 08 03:31:12.508740 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.508724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.517587 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.517519    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:31:12.519703 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.519676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:31:12.589712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.589678    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52ffl\" (UniqueName: \"kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl\") pod \"4465aefb-0724-4a5e-82ce-06c1569fef66\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") "
Jul 08 03:31:12.589900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.589758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume\") pod \"4465aefb-0724-4a5e-82ce-06c1569fef66\" (UID: \"4465aefb-0724-4a5e-82ce-06c1569fef66\") "
Jul 08 03:31:12.589900 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.589884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.589983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.589926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc87z\" (UniqueName: \"kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.590190 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.590164    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4465aefb-0724-4a5e-82ce-06c1569fef66" (UID: "4465aefb-0724-4a5e-82ce-06c1569fef66"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:31:12.591908 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.591885    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl" (OuterVolumeSpecName: "kube-api-access-52ffl") pod "4465aefb-0724-4a5e-82ce-06c1569fef66" (UID: "4465aefb-0724-4a5e-82ce-06c1569fef66"). InnerVolumeSpecName "kube-api-access-52ffl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:31:12.690638 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.690601    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.690810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.690685    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hc87z\" (UniqueName: \"kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.690810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.690750    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52ffl\" (UniqueName: \"kubernetes.io/projected/4465aefb-0724-4a5e-82ce-06c1569fef66-kube-api-access-52ffl\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:31:12.690810 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.690766    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4465aefb-0724-4a5e-82ce-06c1569fef66-extensions-socket-volume\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:31:12.690962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.690942    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.699002 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.698976    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc87z\" (UniqueName: \"kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z\") pod \"kuadrant-operator-controller-manager-86878c769f-9rsls\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.823763 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.823730    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:12.964744 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:12.964719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:31:12.966575 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:31:12.966547    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b986fb7_eb20_4d55_b74b_4e99d7df2282.slice/crio-2f35618d942be4837b1cd5ee2fd50580ffbff8e2438700a047b63cd9d78fbff6 WatchSource:0}: Error finding container 2f35618d942be4837b1cd5ee2fd50580ffbff8e2438700a047b63cd9d78fbff6: Status 404 returned error can't find the container with id 2f35618d942be4837b1cd5ee2fd50580ffbff8e2438700a047b63cd9d78fbff6
Jul 08 03:31:13.365643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.365600    2563 generic.go:356] "Generic (PLEG): container finished" podID="4465aefb-0724-4a5e-82ce-06c1569fef66" containerID="954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455" exitCode=0
Jul 08 03:31:13.365643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.365644    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"
Jul 08 03:31:13.366183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.365641    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" event={"ID":"4465aefb-0724-4a5e-82ce-06c1569fef66","Type":"ContainerDied","Data":"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"}
Jul 08 03:31:13.366183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.365695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt" event={"ID":"4465aefb-0724-4a5e-82ce-06c1569fef66","Type":"ContainerDied","Data":"fffff13fd70e23c68f08cd6b1bd5d829c51259fd65ef6fbe048ce5fccd994b62"}
Jul 08 03:31:13.366183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.365718    2563 scope.go:117] "RemoveContainer" containerID="954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"
Jul 08 03:31:13.367496 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.367471    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" event={"ID":"0b986fb7-eb20-4d55-b74b-4e99d7df2282","Type":"ContainerStarted","Data":"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"}
Jul 08 03:31:13.367630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.367514    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" event={"ID":"0b986fb7-eb20-4d55-b74b-4e99d7df2282","Type":"ContainerStarted","Data":"2f35618d942be4837b1cd5ee2fd50580ffbff8e2438700a047b63cd9d78fbff6"}
Jul 08 03:31:13.367630 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.367539    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:13.375104 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.375086    2563 scope.go:117] "RemoveContainer" containerID="954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"
Jul 08 03:31:13.375340 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:31:13.375323    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455\": container with ID starting with 954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455 not found: ID does not exist" containerID="954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"
Jul 08 03:31:13.375408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.375348    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455"} err="failed to get container status \"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455\": rpc error: code = NotFound desc = could not find container \"954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455\": container with ID starting with 954d2f4c9269d89de604fbe4b9a4b9242140de173e4265d51ab533cf23bc5455 not found: ID does not exist"
Jul 08 03:31:13.403505 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.403459    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" podStartSLOduration=1.403443618 podStartE2EDuration="1.403443618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:13.401801057 +0000 UTC m=+684.510146996" watchObservedRunningTime="2026-07-08 03:31:13.403443618 +0000 UTC m=+684.511789557"
Jul 08 03:31:13.435074 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.435039    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:31:13.445244 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:13.445215    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-tr9nt"]
Jul 08 03:31:15.384404 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:15.384371    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4465aefb-0724-4a5e-82ce-06c1569fef66" path="/var/lib/kubelet/pods/4465aefb-0724-4a5e-82ce-06c1569fef66/volumes"
Jul 08 03:31:24.376830 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:24.376797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:31:32.637277 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.637217    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:32.637837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.637710    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4465aefb-0724-4a5e-82ce-06c1569fef66" containerName="manager"
Jul 08 03:31:32.637837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.637727    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4465aefb-0724-4a5e-82ce-06c1569fef66" containerName="manager"
Jul 08 03:31:32.637837 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.637796    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4465aefb-0724-4a5e-82ce-06c1569fef66" containerName="manager"
Jul 08 03:31:32.642192 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.642174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.644357 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.644334    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 03:31:32.644459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.644340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7mzxj\""
Jul 08 03:31:32.649005 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.648982    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:32.735399 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.735368    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:32.768117 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.768085    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.768117 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.768119    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wkvs\" (UniqueName: \"kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.869507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.869465    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.869507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.869501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wkvs\" (UniqueName: \"kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.870194 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.870172    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.878363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.878342    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wkvs\" (UniqueName: \"kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs\") pod \"limitador-limitador-64c94fd64f-z5s6q\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:32.953946 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:32.953864    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:33.097995 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.097971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:33.100922 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:31:33.100893    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a0dee3_ff93_4899_b9f9_2b7d7314bce6.slice/crio-407583a0f4129a87dd3befac6d916a3013e08fc28ad7b68d771376a9f8962b98 WatchSource:0}: Error finding container 407583a0f4129a87dd3befac6d916a3013e08fc28ad7b68d771376a9f8962b98: Status 404 returned error can't find the container with id 407583a0f4129a87dd3befac6d916a3013e08fc28ad7b68d771376a9f8962b98
Jul 08 03:31:33.440529 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.440490    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:33.445438 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.445414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:33.447879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.447853    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fhl8r\""
Jul 08 03:31:33.449600 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.449576    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:33.477587 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.477560    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" event={"ID":"66a0dee3-ff93-4899-b9f9-2b7d7314bce6","Type":"ContainerStarted","Data":"407583a0f4129a87dd3befac6d916a3013e08fc28ad7b68d771376a9f8962b98"}
Jul 08 03:31:33.575604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.575573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvvds\" (UniqueName: \"kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds\") pod \"authorino-58cd955675-ddxkn\" (UID: \"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0\") " pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:33.580420 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.580396    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:31:33.583901 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.583885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:31:33.590066 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.590043    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:31:33.676608 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.676575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvvds\" (UniqueName: \"kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds\") pod \"authorino-58cd955675-ddxkn\" (UID: \"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0\") " pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:33.677001 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.676680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4spv9\" (UniqueName: \"kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9\") pod \"authorino-75d556bd77-pg4wl\" (UID: \"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6\") " pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:31:33.685266 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.685244    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvvds\" (UniqueName: \"kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds\") pod \"authorino-58cd955675-ddxkn\" (UID: \"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0\") " pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:33.756499 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.756434    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:33.777348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.777324    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4spv9\" (UniqueName: \"kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9\") pod \"authorino-75d556bd77-pg4wl\" (UID: \"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6\") " pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:31:33.785752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.785726    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4spv9\" (UniqueName: \"kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9\") pod \"authorino-75d556bd77-pg4wl\" (UID: \"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6\") " pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:31:33.889471 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.889380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:33.892429 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:31:33.892264    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72f0c9ef_c552_41d2_b5ba_20b35ba12cd0.slice/crio-bef83e707c78deea9c4c1214b235204c168770586a5400da27666f5fcf77cdd2 WatchSource:0}: Error finding container bef83e707c78deea9c4c1214b235204c168770586a5400da27666f5fcf77cdd2: Status 404 returned error can't find the container with id bef83e707c78deea9c4c1214b235204c168770586a5400da27666f5fcf77cdd2
Jul 08 03:31:33.892705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:33.892683    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:31:34.079573 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:34.078909    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:31:34.082697 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:31:34.082645    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6c192cf_4aa9_47df_82b7_7d8f174a6ca6.slice/crio-6e8e923a7237c66315f1f7d20882bf213516a9f8fc1a6d463bdbe069369ce1a9 WatchSource:0}: Error finding container 6e8e923a7237c66315f1f7d20882bf213516a9f8fc1a6d463bdbe069369ce1a9: Status 404 returned error can't find the container with id 6e8e923a7237c66315f1f7d20882bf213516a9f8fc1a6d463bdbe069369ce1a9
Jul 08 03:31:34.485956 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:34.485887    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pg4wl" event={"ID":"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6","Type":"ContainerStarted","Data":"6e8e923a7237c66315f1f7d20882bf213516a9f8fc1a6d463bdbe069369ce1a9"}
Jul 08 03:31:34.487307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:34.487278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ddxkn" event={"ID":"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0","Type":"ContainerStarted","Data":"bef83e707c78deea9c4c1214b235204c168770586a5400da27666f5fcf77cdd2"}
Jul 08 03:31:39.526330 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.526293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" event={"ID":"66a0dee3-ff93-4899-b9f9-2b7d7314bce6","Type":"ContainerStarted","Data":"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"}
Jul 08 03:31:39.526825 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.526406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:39.527898 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.527878    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pg4wl" event={"ID":"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6","Type":"ContainerStarted","Data":"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"}
Jul 08 03:31:39.529394 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.529369    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ddxkn" event={"ID":"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0","Type":"ContainerStarted","Data":"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"}
Jul 08 03:31:39.542061 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.542015    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" podStartSLOduration=2.1780444389999998 podStartE2EDuration="7.542001294s" totalImagesPullingTime="5.363956855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:39.541082431 +0000 UTC m=+710.649428366" watchObservedRunningTime="2026-07-08 03:31:39.542001294 +0000 UTC m=+710.650347233"
Jul 08 03:31:39.554400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.554353    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ddxkn" podStartSLOduration=1.9885202450000001 podStartE2EDuration="6.554339671s" totalImagesPullingTime="4.565819426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:39.553646306 +0000 UTC m=+710.661992247" watchObservedRunningTime="2026-07-08 03:31:39.554339671 +0000 UTC m=+710.662685610"
Jul 08 03:31:39.566167 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.566104    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-pg4wl" podStartSLOduration=2.22450317 podStartE2EDuration="6.566090009s" totalImagesPullingTime="4.341586839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:39.566049323 +0000 UTC m=+710.674395255" watchObservedRunningTime="2026-07-08 03:31:39.566090009 +0000 UTC m=+710.674435950"
Jul 08 03:31:39.594420 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:39.594336    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:41.540183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:41.540116    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ddxkn" podUID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" containerName="authorino" containerID="cri-o://ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6" gracePeriod=30
Jul 08 03:31:41.789005 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:41.788978    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:41.954938 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:41.954900    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvvds\" (UniqueName: \"kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds\") pod \"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0\" (UID: \"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0\") "
Jul 08 03:31:41.957103 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:41.957075    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds" (OuterVolumeSpecName: "kube-api-access-kvvds") pod "72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" (UID: "72f0c9ef-c552-41d2-b5ba-20b35ba12cd0"). InnerVolumeSpecName "kube-api-access-kvvds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:31:42.056278 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.056227    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvvds\" (UniqueName: \"kubernetes.io/projected/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0-kube-api-access-kvvds\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:31:42.547307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.547269    2563 generic.go:356] "Generic (PLEG): container finished" podID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" containerID="ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6" exitCode=0
Jul 08 03:31:42.547751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.547317    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ddxkn"
Jul 08 03:31:42.547751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.547334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ddxkn" event={"ID":"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0","Type":"ContainerDied","Data":"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"}
Jul 08 03:31:42.547751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.547372    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ddxkn" event={"ID":"72f0c9ef-c552-41d2-b5ba-20b35ba12cd0","Type":"ContainerDied","Data":"bef83e707c78deea9c4c1214b235204c168770586a5400da27666f5fcf77cdd2"}
Jul 08 03:31:42.547751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.547394    2563 scope.go:117] "RemoveContainer" containerID="ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"
Jul 08 03:31:42.556794 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.556773    2563 scope.go:117] "RemoveContainer" containerID="ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"
Jul 08 03:31:42.557043 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:31:42.557024    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6\": container with ID starting with ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6 not found: ID does not exist" containerID="ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"
Jul 08 03:31:42.557093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.557050    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6"} err="failed to get container status \"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6\": rpc error: code = NotFound desc = could not find container \"ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6\": container with ID starting with ea6ad0e77cf66791e28c7ac76bc26772f88a78899c9112cefd1be947342437a6 not found: ID does not exist"
Jul 08 03:31:42.569035 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.569008    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:42.576410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:42.576391    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ddxkn"]
Jul 08 03:31:43.385972 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:43.385941    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" path="/var/lib/kubelet/pods/72f0c9ef-c552-41d2-b5ba-20b35ba12cd0/volumes"
Jul 08 03:31:47.431358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:47.431315    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:47.431876 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:47.431669    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" podUID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" containerName="limitador" containerID="cri-o://b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc" gracePeriod=30
Jul 08 03:31:47.433902 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:47.433883    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:47.992299 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:47.992232    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:48.003720 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.003691    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file\") pod \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") "
Jul 08 03:31:48.003847 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.003766    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8wkvs\" (UniqueName: \"kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs\") pod \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\" (UID: \"66a0dee3-ff93-4899-b9f9-2b7d7314bce6\") "
Jul 08 03:31:48.004101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.004075    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file" (OuterVolumeSpecName: "config-file") pod "66a0dee3-ff93-4899-b9f9-2b7d7314bce6" (UID: "66a0dee3-ff93-4899-b9f9-2b7d7314bce6"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:31:48.005868 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.005841    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs" (OuterVolumeSpecName: "kube-api-access-8wkvs") pod "66a0dee3-ff93-4899-b9f9-2b7d7314bce6" (UID: "66a0dee3-ff93-4899-b9f9-2b7d7314bce6"). InnerVolumeSpecName "kube-api-access-8wkvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:31:48.104836 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.104793    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-config-file\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:31:48.104836 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.104833    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8wkvs\" (UniqueName: \"kubernetes.io/projected/66a0dee3-ff93-4899-b9f9-2b7d7314bce6-kube-api-access-8wkvs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:31:48.582253 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.582220    2563 generic.go:356] "Generic (PLEG): container finished" podID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" containerID="b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc" exitCode=0
Jul 08 03:31:48.582734 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.582264    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"
Jul 08 03:31:48.582734 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.582278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" event={"ID":"66a0dee3-ff93-4899-b9f9-2b7d7314bce6","Type":"ContainerDied","Data":"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"}
Jul 08 03:31:48.582734 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.582319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5s6q" event={"ID":"66a0dee3-ff93-4899-b9f9-2b7d7314bce6","Type":"ContainerDied","Data":"407583a0f4129a87dd3befac6d916a3013e08fc28ad7b68d771376a9f8962b98"}
Jul 08 03:31:48.582734 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.582340    2563 scope.go:117] "RemoveContainer" containerID="b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"
Jul 08 03:31:48.594182 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.594163    2563 scope.go:117] "RemoveContainer" containerID="b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"
Jul 08 03:31:48.594452 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:31:48.594431    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc\": container with ID starting with b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc not found: ID does not exist" containerID="b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"
Jul 08 03:31:48.594510 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.594460    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc"} err="failed to get container status \"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc\": rpc error: code = NotFound desc = could not find container \"b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc\": container with ID starting with b7f34ba8c558c03f194d7568ac23a64e815422cd40eed757264061f5a67628bc not found: ID does not exist"
Jul 08 03:31:48.608683 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.608654    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:48.611788 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:48.611766    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5s6q"]
Jul 08 03:31:49.384889 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:49.384862    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" path="/var/lib/kubelet/pods/66a0dee3-ff93-4899-b9f9-2b7d7314bce6/volumes"
Jul 08 03:31:51.889036 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889003    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-2j4bv"]
Jul 08 03:31:51.889433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889381    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" containerName="authorino"
Jul 08 03:31:51.889433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889392    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" containerName="authorino"
Jul 08 03:31:51.889433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889403    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" containerName="limitador"
Jul 08 03:31:51.889433 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889407    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" containerName="limitador"
Jul 08 03:31:51.889560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889473    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="66a0dee3-ff93-4899-b9f9-2b7d7314bce6" containerName="limitador"
Jul 08 03:31:51.889560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.889481    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72f0c9ef-c552-41d2-b5ba-20b35ba12cd0" containerName="authorino"
Jul 08 03:31:51.893813 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.893796    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:51.896354 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.896326    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 08 03:31:51.896465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.896434    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-z4dt7\""
Jul 08 03:31:51.902400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.902377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-2j4bv"]
Jul 08 03:31:51.933020 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.932994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53cbffe1-3a17-4d0b-b882-87ef081af3fd-data\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:51.933184 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:51.933029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/53cbffe1-3a17-4d0b-b882-87ef081af3fd-kube-api-access-rkvxf\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.034451 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.034398    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53cbffe1-3a17-4d0b-b882-87ef081af3fd-data\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.034733 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.034713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/53cbffe1-3a17-4d0b-b882-87ef081af3fd-kube-api-access-rkvxf\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.040459 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.035792    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/53cbffe1-3a17-4d0b-b882-87ef081af3fd-data\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.043892 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.043866    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/53cbffe1-3a17-4d0b-b882-87ef081af3fd-kube-api-access-rkvxf\") pod \"postgres-69fb4594cf-2j4bv\" (UID: \"53cbffe1-3a17-4d0b-b882-87ef081af3fd\") " pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.206101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.206004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:52.343472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.343445    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-2j4bv"]
Jul 08 03:31:52.345258 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:31:52.345231    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53cbffe1_3a17_4d0b_b882_87ef081af3fd.slice/crio-069b1cf00c2d2b893ab99915938bfc9d1dd06fefc88365abbc14729250ab5b4c WatchSource:0}: Error finding container 069b1cf00c2d2b893ab99915938bfc9d1dd06fefc88365abbc14729250ab5b4c: Status 404 returned error can't find the container with id 069b1cf00c2d2b893ab99915938bfc9d1dd06fefc88365abbc14729250ab5b4c
Jul 08 03:31:52.605481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:52.605440    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-2j4bv" event={"ID":"53cbffe1-3a17-4d0b-b882-87ef081af3fd","Type":"ContainerStarted","Data":"069b1cf00c2d2b893ab99915938bfc9d1dd06fefc88365abbc14729250ab5b4c"}
Jul 08 03:31:58.647649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:58.647614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-2j4bv" event={"ID":"53cbffe1-3a17-4d0b-b882-87ef081af3fd","Type":"ContainerStarted","Data":"16ef77791daff87bfd29bb0d3d11be86a08c9cd3ee4d016ee510de329d706fa3"}
Jul 08 03:31:58.648050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:58.647781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:31:58.665808 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:31:58.665746    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-2j4bv" podStartSLOduration=2.153517305 podStartE2EDuration="7.665731909s" totalImagesPullingTime="5.512214604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:31:58.663496702 +0000 UTC m=+729.771842643" watchObservedRunningTime="2026-07-08 03:31:58.665731909 +0000 UTC m=+729.774077911"
Jul 08 03:32:04.683451 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:04.683419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-2j4bv"
Jul 08 03:32:05.196587 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.196555    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:05.200488 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.200469    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:05.207962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.207671    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:05.261441 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.261403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwr4c\" (UniqueName: \"kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c\") pod \"authorino-5d4b67c446-vz4n9\" (UID: \"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6\") " pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:05.361917 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.361879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwr4c\" (UniqueName: \"kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c\") pod \"authorino-5d4b67c446-vz4n9\" (UID: \"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6\") " pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:05.370784 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.370751    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwr4c\" (UniqueName: \"kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c\") pod \"authorino-5d4b67c446-vz4n9\" (UID: \"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6\") " pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:05.411457 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.411417    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:05.411688 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.411675    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:05.436748 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.436717    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7fcc84b49c-7hmn6"]
Jul 08 03:32:05.439911 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.439888    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.446920 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.446849    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7fcc84b49c-7hmn6"]
Jul 08 03:32:05.550431 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.550401    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:05.552097 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:05.552070    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd57696c8_d67c_4f3b_ab58_d2f1d1049ae6.slice/crio-d37a1b792c10a8fe3f6742661f289cad2795aa6afa640e981ac042b15fabcd53 WatchSource:0}: Error finding container d37a1b792c10a8fe3f6742661f289cad2795aa6afa640e981ac042b15fabcd53: Status 404 returned error can't find the container with id d37a1b792c10a8fe3f6742661f289cad2795aa6afa640e981ac042b15fabcd53
Jul 08 03:32:05.564595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.564570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvb94\" (UniqueName: \"kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94\") pod \"authorino-7fcc84b49c-7hmn6\" (UID: \"bc1b13c7-aa71-4820-b466-d650003b0051\") " pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.635214 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.635180    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fcc84b49c-7hmn6"]
Jul 08 03:32:05.635489 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:05.635469    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-rvb94], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7fcc84b49c-7hmn6" podUID="bc1b13c7-aa71-4820-b466-d650003b0051"
Jul 08 03:32:05.661338 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.661302    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:32:05.665203 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.665175    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvb94\" (UniqueName: \"kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94\") pod \"authorino-7fcc84b49c-7hmn6\" (UID: \"bc1b13c7-aa71-4820-b466-d650003b0051\") " pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.666819 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.666801    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.669649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.669329    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 08 03:32:05.672829 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.672723    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:32:05.677374 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.677353    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvb94\" (UniqueName: \"kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94\") pod \"authorino-7fcc84b49c-7hmn6\" (UID: \"bc1b13c7-aa71-4820-b466-d650003b0051\") " pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.688706 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.688668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.688706 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.688661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" event={"ID":"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6","Type":"ContainerStarted","Data":"d37a1b792c10a8fe3f6742661f289cad2795aa6afa640e981ac042b15fabcd53"}
Jul 08 03:32:05.700126 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.700074    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:05.766121 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.766090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rvb94\" (UniqueName: \"kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94\") pod \"bc1b13c7-aa71-4820-b466-d650003b0051\" (UID: \"bc1b13c7-aa71-4820-b466-d650003b0051\") "
Jul 08 03:32:05.766297 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.766273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gf4kr\" (UniqueName: \"kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.766426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.766404    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.768303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.768283    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94" (OuterVolumeSpecName: "kube-api-access-rvb94") pod "bc1b13c7-aa71-4820-b466-d650003b0051" (UID: "bc1b13c7-aa71-4820-b466-d650003b0051"). InnerVolumeSpecName "kube-api-access-rvb94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:32:05.866981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.866940    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.867206 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.867001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gf4kr\" (UniqueName: \"kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.867206 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.867153    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rvb94\" (UniqueName: \"kubernetes.io/projected/bc1b13c7-aa71-4820-b466-d650003b0051-kube-api-access-rvb94\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:32:05.869548 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.869524    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.874645 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.874626    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gf4kr\" (UniqueName: \"kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr\") pod \"authorino-66446d9d49-b48gv\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") " pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:05.992025 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:05.992004    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:32:06.144163 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.144120    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:32:06.146303 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:06.146277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod157e92de_b877_4572_9fcc_7341f8747eb0.slice/crio-7af75f1d995d41f60e182f38f18d0de4050796e2fe40979fcb5f3a9381dafda2 WatchSource:0}: Error finding container 7af75f1d995d41f60e182f38f18d0de4050796e2fe40979fcb5f3a9381dafda2: Status 404 returned error can't find the container with id 7af75f1d995d41f60e182f38f18d0de4050796e2fe40979fcb5f3a9381dafda2
Jul 08 03:32:06.695393 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.695298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" event={"ID":"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6","Type":"ContainerStarted","Data":"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"}
Jul 08 03:32:06.695393 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.695307    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" podUID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" containerName="authorino" containerID="cri-o://1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a" gracePeriod=30
Jul 08 03:32:06.696667 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.696649    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7fcc84b49c-7hmn6"
Jul 08 03:32:06.696769 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.696648    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66446d9d49-b48gv" event={"ID":"157e92de-b877-4572-9fcc-7341f8747eb0","Type":"ContainerStarted","Data":"7af75f1d995d41f60e182f38f18d0de4050796e2fe40979fcb5f3a9381dafda2"}
Jul 08 03:32:06.712016 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.711975    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" podStartSLOduration=1.299925368 podStartE2EDuration="1.711961098s" totalImagesPullingTime="412.03573ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:06.71000495 +0000 UTC m=+737.818350890" watchObservedRunningTime="2026-07-08 03:32:06.711961098 +0000 UTC m=+737.820307037"
Jul 08 03:32:06.732698 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.732659    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7fcc84b49c-7hmn6"]
Jul 08 03:32:06.741403 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.741377    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7fcc84b49c-7hmn6"]
Jul 08 03:32:06.988565 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:06.988542    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:07.078234 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.078191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dwr4c\" (UniqueName: \"kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c\") pod \"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6\" (UID: \"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6\") "
Jul 08 03:32:07.080640 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.080599    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c" (OuterVolumeSpecName: "kube-api-access-dwr4c") pod "d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" (UID: "d57696c8-d67c-4f3b-ab58-d2f1d1049ae6"). InnerVolumeSpecName "kube-api-access-dwr4c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:32:07.179777 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.179743    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dwr4c\" (UniqueName: \"kubernetes.io/projected/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6-kube-api-access-dwr4c\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:32:07.385310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.385281    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bc1b13c7-aa71-4820-b466-d650003b0051" path="/var/lib/kubelet/pods/bc1b13c7-aa71-4820-b466-d650003b0051/volumes"
Jul 08 03:32:07.703477 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.703386    2563 generic.go:356] "Generic (PLEG): container finished" podID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" containerID="1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a" exitCode=0
Jul 08 03:32:07.703477 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.703442    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vz4n9"
Jul 08 03:32:07.703477 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.703444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" event={"ID":"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6","Type":"ContainerDied","Data":"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"}
Jul 08 03:32:07.703994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.703494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-vz4n9" event={"ID":"d57696c8-d67c-4f3b-ab58-d2f1d1049ae6","Type":"ContainerDied","Data":"d37a1b792c10a8fe3f6742661f289cad2795aa6afa640e981ac042b15fabcd53"}
Jul 08 03:32:07.703994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.703513    2563 scope.go:117] "RemoveContainer" containerID="1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"
Jul 08 03:32:07.705503 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.705474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66446d9d49-b48gv" event={"ID":"157e92de-b877-4572-9fcc-7341f8747eb0","Type":"ContainerStarted","Data":"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"}
Jul 08 03:32:07.715251 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.715234    2563 scope.go:117] "RemoveContainer" containerID="1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"
Jul 08 03:32:07.715521 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:07.715502    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a\": container with ID starting with 1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a not found: ID does not exist" containerID="1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"
Jul 08 03:32:07.715571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.715529    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a"} err="failed to get container status \"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a\": rpc error: code = NotFound desc = could not find container \"1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a\": container with ID starting with 1f39acdc8d90612cde1bf9c28b0b1c9f13210bb177fd21b4f6e136e66dea418a not found: ID does not exist"
Jul 08 03:32:07.729020 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.728980    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-66446d9d49-b48gv" podStartSLOduration=1.9680352540000001 podStartE2EDuration="2.728967237s" totalImagesPullingTime="760.931983ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:07.726989066 +0000 UTC m=+738.835335007" watchObservedRunningTime="2026-07-08 03:32:07.728967237 +0000 UTC m=+738.837313178"
Jul 08 03:32:07.744902 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.744874    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:07.750727 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.750702    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vz4n9"]
Jul 08 03:32:07.759680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.759658    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:32:07.759865 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:07.759842    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-pg4wl" podUID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" containerName="authorino" containerID="cri-o://cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc" gracePeriod=30
Jul 08 03:32:08.052132 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.052106    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:32:08.189524 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.189492    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4spv9\" (UniqueName: \"kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9\") pod \"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6\" (UID: \"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6\") "
Jul 08 03:32:08.191676 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.191649    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9" (OuterVolumeSpecName: "kube-api-access-4spv9") pod "c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" (UID: "c6c192cf-4aa9-47df-82b7-7d8f174a6ca6"). InnerVolumeSpecName "kube-api-access-4spv9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:32:08.290977 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.290886    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4spv9\" (UniqueName: \"kubernetes.io/projected/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6-kube-api-access-4spv9\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:32:08.712253 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.712217    2563 generic.go:356] "Generic (PLEG): container finished" podID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" containerID="cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc" exitCode=0
Jul 08 03:32:08.712697 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.712264    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-pg4wl"
Jul 08 03:32:08.712697 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.712304    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pg4wl" event={"ID":"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6","Type":"ContainerDied","Data":"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"}
Jul 08 03:32:08.712697 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.712351    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-pg4wl" event={"ID":"c6c192cf-4aa9-47df-82b7-7d8f174a6ca6","Type":"ContainerDied","Data":"6e8e923a7237c66315f1f7d20882bf213516a9f8fc1a6d463bdbe069369ce1a9"}
Jul 08 03:32:08.712697 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.712377    2563 scope.go:117] "RemoveContainer" containerID="cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"
Jul 08 03:32:08.721583 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.721562    2563 scope.go:117] "RemoveContainer" containerID="cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"
Jul 08 03:32:08.721849 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:08.721832    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc\": container with ID starting with cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc not found: ID does not exist" containerID="cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"
Jul 08 03:32:08.721915 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.721855    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc"} err="failed to get container status \"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc\": rpc error: code = NotFound desc = could not find container \"cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc\": container with ID starting with cd195fa90269164e7fa6fa7cc61b22a1f2e139b64d1d4baa2592609e61e6cfcc not found: ID does not exist"
Jul 08 03:32:08.733274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.733243    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:32:08.735712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:08.735688    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-pg4wl"]
Jul 08 03:32:09.385442 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:09.385410    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" path="/var/lib/kubelet/pods/c6c192cf-4aa9-47df-82b7-7d8f174a6ca6/volumes"
Jul 08 03:32:09.385746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:09.385733    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" path="/var/lib/kubelet/pods/d57696c8-d67c-4f3b-ab58-d2f1d1049ae6/volumes"
Jul 08 03:32:11.925246 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925216    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925593    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" containerName="authorino"
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925603    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" containerName="authorino"
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925615    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" containerName="authorino"
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925621    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" containerName="authorino"
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925679    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6c192cf-4aa9-47df-82b7-7d8f174a6ca6" containerName="authorino"
Jul 08 03:32:11.925705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.925689    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d57696c8-d67c-4f3b-ab58-d2f1d1049ae6" containerName="authorino"
Jul 08 03:32:11.935802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.935777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:11.939623 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.939595    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 03:32:11.942580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.942552    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 03:32:11.942580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.942570    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tvjh8\""
Jul 08 03:32:11.945934 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:11.945908    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:12.025039 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.025005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q2b5\" (UniqueName: \"kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.025248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.025070    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.096696 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.096658    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:32:12.099183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.099130    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.109421 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.109395    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:32:12.126363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.126329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q2b5\" (UniqueName: \"kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.126507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.126423    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.129242 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.129218    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.134004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.133979    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q2b5\" (UniqueName: \"kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5\") pod \"maas-controller-588585c46d-j29l4\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") " pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.227955 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.227872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.227955 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.227913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4thv\" (UniqueName: \"kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.248747 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.248715    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:12.329560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.329061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.329560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.329123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4thv\" (UniqueName: \"kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.334045 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.332905    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.339191 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.339130    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4thv\" (UniqueName: \"kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv\") pod \"maas-controller-b857d6d59-bn2kv\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") " pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.410304 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.410271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:12.415123 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.415096    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:12.416131 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:12.416107    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5df5fe4_0828_44aa_9316_789d399710b6.slice/crio-0bf65886f52d60c1e0c9b73234074d12118e2834dfa175736b0b4972d27d07a6 WatchSource:0}: Error finding container 0bf65886f52d60c1e0c9b73234074d12118e2834dfa175736b0b4972d27d07a6: Status 404 returned error can't find the container with id 0bf65886f52d60c1e0c9b73234074d12118e2834dfa175736b0b4972d27d07a6
Jul 08 03:32:12.549286 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.549262    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:32:12.550533 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:12.550509    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4051b7c_6aea_4449_b959_7c231ae2977a.slice/crio-eb381dbe3a3a9a98d5047f359630fd45fcd732c5c5d17782a994121044785e1d WatchSource:0}: Error finding container eb381dbe3a3a9a98d5047f359630fd45fcd732c5c5d17782a994121044785e1d: Status 404 returned error can't find the container with id eb381dbe3a3a9a98d5047f359630fd45fcd732c5c5d17782a994121044785e1d
Jul 08 03:32:12.738100 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.738066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-bn2kv" event={"ID":"e4051b7c-6aea-4449-b959-7c231ae2977a","Type":"ContainerStarted","Data":"eb381dbe3a3a9a98d5047f359630fd45fcd732c5c5d17782a994121044785e1d"}
Jul 08 03:32:12.739554 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:12.739521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588585c46d-j29l4" event={"ID":"d5df5fe4-0828-44aa-9316-789d399710b6","Type":"ContainerStarted","Data":"0bf65886f52d60c1e0c9b73234074d12118e2834dfa175736b0b4972d27d07a6"}
Jul 08 03:32:14.756416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.756323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-bn2kv" event={"ID":"e4051b7c-6aea-4449-b959-7c231ae2977a","Type":"ContainerStarted","Data":"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"}
Jul 08 03:32:14.756416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.756388    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:14.757931 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.757908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588585c46d-j29l4" event={"ID":"d5df5fe4-0828-44aa-9316-789d399710b6","Type":"ContainerStarted","Data":"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"}
Jul 08 03:32:14.758050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.757988    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:14.782230 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.782188    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b857d6d59-bn2kv" podStartSLOduration=0.822614245 podStartE2EDuration="2.782168465s" totalImagesPullingTime="1.95955422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:14.780220061 +0000 UTC m=+745.888566001" watchObservedRunningTime="2026-07-08 03:32:14.782168465 +0000 UTC m=+745.890514410"
Jul 08 03:32:14.803568 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:14.803522    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-588585c46d-j29l4" podStartSLOduration=1.72419447 podStartE2EDuration="3.803501946s" totalImagesPullingTime="2.079307476s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:14.801408084 +0000 UTC m=+745.909754025" watchObservedRunningTime="2026-07-08 03:32:14.803501946 +0000 UTC m=+745.911847887"
Jul 08 03:32:25.768873 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:25.768836    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:32:25.769379 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:25.768916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:25.830572 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:25.830541    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:25.830765 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:25.830720    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-588585c46d-j29l4" podUID="d5df5fe4-0828-44aa-9316-789d399710b6" containerName="manager" containerID="cri-o://f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a" gracePeriod=30
Jul 08 03:32:26.080654 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.080625    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:26.169253 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.169221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q2b5\" (UniqueName: \"kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5\") pod \"d5df5fe4-0828-44aa-9316-789d399710b6\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") "
Jul 08 03:32:26.169457 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.169275    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert\") pod \"d5df5fe4-0828-44aa-9316-789d399710b6\" (UID: \"d5df5fe4-0828-44aa-9316-789d399710b6\") "
Jul 08 03:32:26.171588 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.171557    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d5df5fe4-0828-44aa-9316-789d399710b6" (UID: "d5df5fe4-0828-44aa-9316-789d399710b6"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:32:26.171700 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.171677    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5" (OuterVolumeSpecName: "kube-api-access-9q2b5") pod "d5df5fe4-0828-44aa-9316-789d399710b6" (UID: "d5df5fe4-0828-44aa-9316-789d399710b6"). InnerVolumeSpecName "kube-api-access-9q2b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:32:26.270307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.270267    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q2b5\" (UniqueName: \"kubernetes.io/projected/d5df5fe4-0828-44aa-9316-789d399710b6-kube-api-access-9q2b5\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:32:26.270307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.270301    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d5df5fe4-0828-44aa-9316-789d399710b6-webhook-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:32:26.827920 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.827885    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5df5fe4-0828-44aa-9316-789d399710b6" containerID="f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a" exitCode=0
Jul 08 03:32:26.828388 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.827937    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588585c46d-j29l4"
Jul 08 03:32:26.828388 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.827980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588585c46d-j29l4" event={"ID":"d5df5fe4-0828-44aa-9316-789d399710b6","Type":"ContainerDied","Data":"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"}
Jul 08 03:32:26.828388 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.828037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588585c46d-j29l4" event={"ID":"d5df5fe4-0828-44aa-9316-789d399710b6","Type":"ContainerDied","Data":"0bf65886f52d60c1e0c9b73234074d12118e2834dfa175736b0b4972d27d07a6"}
Jul 08 03:32:26.828388 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.828070    2563 scope.go:117] "RemoveContainer" containerID="f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"
Jul 08 03:32:26.838291 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.838100    2563 scope.go:117] "RemoveContainer" containerID="f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"
Jul 08 03:32:26.838508 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:26.838491    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a\": container with ID starting with f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a not found: ID does not exist" containerID="f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"
Jul 08 03:32:26.838552 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.838516    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a"} err="failed to get container status \"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a\": rpc error: code = NotFound desc = could not find container \"f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a\": container with ID starting with f7b0233f0128241fa44a84625c9784d9984596b2395a2d44f2fccf2e7b861a2a not found: ID does not exist"
Jul 08 03:32:26.849794 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.849772    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:26.853904 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:26.853884    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-588585c46d-j29l4"]
Jul 08 03:32:27.385656 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:27.385623    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5df5fe4-0828-44aa-9316-789d399710b6" path="/var/lib/kubelet/pods/d5df5fe4-0828-44aa-9316-789d399710b6/volumes"
Jul 08 03:32:46.703598 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.703560    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-bcbd5b4c9-2dd2c"]
Jul 08 03:32:46.704232 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.703954    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5df5fe4-0828-44aa-9316-789d399710b6" containerName="manager"
Jul 08 03:32:46.704232 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.703967    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5df5fe4-0828-44aa-9316-789d399710b6" containerName="manager"
Jul 08 03:32:46.704232 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.704044    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5df5fe4-0828-44aa-9316-789d399710b6" containerName="manager"
Jul 08 03:32:46.707149 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.707117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.709906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.709880    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 08 03:32:46.710049 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.709892    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-2k77n\""
Jul 08 03:32:46.720334 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.720304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bcbd5b4c9-2dd2c"]
Jul 08 03:32:46.851915 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.851881    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.852108 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.851933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d752bb66-9c4f-4ea7-ae8f-31af63170554-openshift-service-ca\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.852108 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.852003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzz2b\" (UniqueName: \"kubernetes.io/projected/d752bb66-9c4f-4ea7-ae8f-31af63170554-kube-api-access-zzz2b\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.952734 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.952698    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.952922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.952748    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d752bb66-9c4f-4ea7-ae8f-31af63170554-openshift-service-ca\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.952922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.952772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzz2b\" (UniqueName: \"kubernetes.io/projected/d752bb66-9c4f-4ea7-ae8f-31af63170554-kube-api-access-zzz2b\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.952922 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:46.952859    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 08 03:32:46.953103 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:32:46.952949    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls podName:d752bb66-9c4f-4ea7-ae8f-31af63170554 nodeName:}" failed. No retries permitted until 2026-07-08 03:32:47.452918092 +0000 UTC m=+778.561264030 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls") pod "maas-api-bcbd5b4c9-2dd2c" (UID: "d752bb66-9c4f-4ea7-ae8f-31af63170554") : secret "maas-api-serving-cert" not found
Jul 08 03:32:46.953457 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.953434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d752bb66-9c4f-4ea7-ae8f-31af63170554-openshift-service-ca\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:46.961063 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:46.960992    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzz2b\" (UniqueName: \"kubernetes.io/projected/d752bb66-9c4f-4ea7-ae8f-31af63170554-kube-api-access-zzz2b\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:47.270579 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.270506    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6rfwn"]
Jul 08 03:32:47.276946 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.276924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.281940 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.281896    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 08 03:32:47.282079 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.281959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-npzdz\""
Jul 08 03:32:47.295906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.295880    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6rfwn"]
Jul 08 03:32:47.355981 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.355949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njfb7\" (UniqueName: \"kubernetes.io/projected/e480dfee-6196-42e2-abc0-9e0b027f4db5-kube-api-access-njfb7\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.356179 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.356018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e480dfee-6196-42e2-abc0-9e0b027f4db5-plugins-config-volume\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.456880 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.456849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e480dfee-6196-42e2-abc0-9e0b027f4db5-plugins-config-volume\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.457076 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.456900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:47.457076 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.456986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njfb7\" (UniqueName: \"kubernetes.io/projected/e480dfee-6196-42e2-abc0-9e0b027f4db5-kube-api-access-njfb7\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.457746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.457718    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e480dfee-6196-42e2-abc0-9e0b027f4db5-plugins-config-volume\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.459506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.459485    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d752bb66-9c4f-4ea7-ae8f-31af63170554-maas-api-tls\") pod \"maas-api-bcbd5b4c9-2dd2c\" (UID: \"d752bb66-9c4f-4ea7-ae8f-31af63170554\") " pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:47.464955 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.464937    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njfb7\" (UniqueName: \"kubernetes.io/projected/e480dfee-6196-42e2-abc0-9e0b027f4db5-kube-api-access-njfb7\") pod \"payload-processing-58f86c7798-6rfwn\" (UID: \"e480dfee-6196-42e2-abc0-9e0b027f4db5\") " pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.589389 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.589359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:47.619468 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.619432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:47.752019 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.751984    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-6rfwn"]
Jul 08 03:32:47.755737 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:47.755695    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode480dfee_6196_42e2_abc0_9e0b027f4db5.slice/crio-64554f692861234ecc8605ae23de8cc2d66c485ac4ca88b23c5c7a67fe18ed18 WatchSource:0}: Error finding container 64554f692861234ecc8605ae23de8cc2d66c485ac4ca88b23c5c7a67fe18ed18: Status 404 returned error can't find the container with id 64554f692861234ecc8605ae23de8cc2d66c485ac4ca88b23c5c7a67fe18ed18
Jul 08 03:32:47.780336 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.780307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-bcbd5b4c9-2dd2c"]
Jul 08 03:32:47.781844 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:32:47.781821    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd752bb66_9c4f_4ea7_ae8f_31af63170554.slice/crio-f57caed96795ce2b75af7f55987c34e4529780b7560410b81153a6c642b7a678 WatchSource:0}: Error finding container f57caed96795ce2b75af7f55987c34e4529780b7560410b81153a6c642b7a678: Status 404 returned error can't find the container with id f57caed96795ce2b75af7f55987c34e4529780b7560410b81153a6c642b7a678
Jul 08 03:32:47.950261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.950178    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn" event={"ID":"e480dfee-6196-42e2-abc0-9e0b027f4db5","Type":"ContainerStarted","Data":"64554f692861234ecc8605ae23de8cc2d66c485ac4ca88b23c5c7a67fe18ed18"}
Jul 08 03:32:47.951466 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:47.951446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c" event={"ID":"d752bb66-9c4f-4ea7-ae8f-31af63170554","Type":"ContainerStarted","Data":"f57caed96795ce2b75af7f55987c34e4529780b7560410b81153a6c642b7a678"}
Jul 08 03:32:49.966274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:49.966180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn" event={"ID":"e480dfee-6196-42e2-abc0-9e0b027f4db5","Type":"ContainerStarted","Data":"e52ac4f97b086fe6095a15d56711d9e3c40adf4e11e2e9b21e66bef7e75fe5a7"}
Jul 08 03:32:49.966669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:49.966293    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:32:49.967773 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:49.967752    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c" event={"ID":"d752bb66-9c4f-4ea7-ae8f-31af63170554","Type":"ContainerStarted","Data":"34d94c0a04d2cec89dce4273140c960c0ae09e753d3d24244097a148a76929ef"}
Jul 08 03:32:49.967886 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:49.967809    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:32:49.985130 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:49.985075    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn" podStartSLOduration=1.5420120389999998 podStartE2EDuration="2.985055985s" totalImagesPullingTime="1.443043946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:49.983655461 +0000 UTC m=+781.092001414" watchObservedRunningTime="2026-07-08 03:32:49.985055985 +0000 UTC m=+781.093401930"
Jul 08 03:32:50.003163 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:50.003080    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c" podStartSLOduration=2.1662593 podStartE2EDuration="4.003061703s" totalImagesPullingTime="1.836802403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:32:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:32:50.002412026 +0000 UTC m=+781.110757966" watchObservedRunningTime="2026-07-08 03:32:50.003061703 +0000 UTC m=+781.111407658"
Jul 08 03:32:55.979416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:32:55.979383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-bcbd5b4c9-2dd2c"
Jul 08 03:33:00.976202 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:00.976125    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-6rfwn"
Jul 08 03:33:28.947058 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.947022    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"]
Jul 08 03:33:28.953541 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.953516    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:28.955678 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.955645    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 03:33:28.955814 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.955687    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 08 03:33:28.956261 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.956239    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tb89x\""
Jul 08 03:33:28.956373 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.956264    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 03:33:28.961410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:28.961371    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"]
Jul 08 03:33:29.041802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.041755    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.041802 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.041811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh44f\" (UniqueName: \"kubernetes.io/projected/efcec4d9-7c63-4550-89f1-58363b298c46-kube-api-access-rh44f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.042023 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.041884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.042023 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.041931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.042023 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.041965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efcec4d9-7c63-4550-89f1-58363b298c46-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.042023 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.042007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143042 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rh44f\" (UniqueName: \"kubernetes.io/projected/efcec4d9-7c63-4550-89f1-58363b298c46-kube-api-access-rh44f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143349 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143349 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143114    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143349 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143186    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efcec4d9-7c63-4550-89f1-58363b298c46-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143349 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143500    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143545    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.143564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.143557    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.145582 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.145548    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efcec4d9-7c63-4550-89f1-58363b298c46-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.145962 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.145941    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efcec4d9-7c63-4550-89f1-58363b298c46-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.151626 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.151601    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh44f\" (UniqueName: \"kubernetes.io/projected/efcec4d9-7c63-4550-89f1-58363b298c46-kube-api-access-rh44f\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk\" (UID: \"efcec4d9-7c63-4550-89f1-58363b298c46\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.268942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.268844    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:29.424167 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:29.424124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"]
Jul 08 03:33:29.426727 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:33:29.426691    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefcec4d9_7c63_4550_89f1_58363b298c46.slice/crio-b3424ef87a1322405d7c61ada5435535e43e33f5ffecb1e5d63b2fcf010ce8d3 WatchSource:0}: Error finding container b3424ef87a1322405d7c61ada5435535e43e33f5ffecb1e5d63b2fcf010ce8d3: Status 404 returned error can't find the container with id b3424ef87a1322405d7c61ada5435535e43e33f5ffecb1e5d63b2fcf010ce8d3
Jul 08 03:33:30.205989 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:30.205953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk" event={"ID":"efcec4d9-7c63-4550-89f1-58363b298c46","Type":"ContainerStarted","Data":"b3424ef87a1322405d7c61ada5435535e43e33f5ffecb1e5d63b2fcf010ce8d3"}
Jul 08 03:33:31.213964 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:31.213924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk" event={"ID":"efcec4d9-7c63-4550-89f1-58363b298c46","Type":"ContainerStarted","Data":"33f409f4a4a90c2326fdf5166db76d943ef5804a7810d48660cf21536586c956"}
Jul 08 03:33:31.233401 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:31.233342    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk" podStartSLOduration=1.9724790410000002 podStartE2EDuration="3.233326568s" totalImagesPullingTime="1.260847527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:33:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:33:31.231291438 +0000 UTC m=+822.339637382" watchObservedRunningTime="2026-07-08 03:33:31.233326568 +0000 UTC m=+822.341672547"
Jul 08 03:33:39.269884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:39.269845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:39.270304 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:39.269916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:39.282479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:39.282450    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:33:40.277363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:33:40.277337    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk"
Jul 08 03:34:49.963811 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:34:49.963695    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:34:49.969604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:34:49.964233    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:34:49.969604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:34:49.968172    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:34:49.969604 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:34:49.968456    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:35:44.748555 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.748517    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76b46ff5d4-rk57q"]
Jul 08 03:35:44.752134 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.752112    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.758957 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.758910    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b46ff5d4-rk57q"]
Jul 08 03:35:44.786924 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.786888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f077e16d-a017-48e7-a15c-f8c1420ef50b-tls-cert\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.787080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.786957    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcd7s\" (UniqueName: \"kubernetes.io/projected/f077e16d-a017-48e7-a15c-f8c1420ef50b-kube-api-access-gcd7s\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.888315 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.888277    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcd7s\" (UniqueName: \"kubernetes.io/projected/f077e16d-a017-48e7-a15c-f8c1420ef50b-kube-api-access-gcd7s\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.888481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.888410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f077e16d-a017-48e7-a15c-f8c1420ef50b-tls-cert\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.891152 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.891105    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/f077e16d-a017-48e7-a15c-f8c1420ef50b-tls-cert\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:44.896422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:44.896398    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcd7s\" (UniqueName: \"kubernetes.io/projected/f077e16d-a017-48e7-a15c-f8c1420ef50b-kube-api-access-gcd7s\") pod \"authorino-76b46ff5d4-rk57q\" (UID: \"f077e16d-a017-48e7-a15c-f8c1420ef50b\") " pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:45.063816 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:45.063777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76b46ff5d4-rk57q"
Jul 08 03:35:45.210288 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:45.210264    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76b46ff5d4-rk57q"]
Jul 08 03:35:45.211605 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:35:45.211575    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf077e16d_a017_48e7_a15c_f8c1420ef50b.slice/crio-38b2e2b275b920976eee124441a2210782d441773cbd31e73f33347a5cbd786a WatchSource:0}: Error finding container 38b2e2b275b920976eee124441a2210782d441773cbd31e73f33347a5cbd786a: Status 404 returned error can't find the container with id 38b2e2b275b920976eee124441a2210782d441773cbd31e73f33347a5cbd786a
Jul 08 03:35:45.213271 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:45.213252    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:35:46.012473 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.012389    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b46ff5d4-rk57q" event={"ID":"f077e16d-a017-48e7-a15c-f8c1420ef50b","Type":"ContainerStarted","Data":"d2d032ad42716293c27f7dcc07776b622e512e5cd5f21ac3f75732aa0d8054a8"}
Jul 08 03:35:46.012473 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.012442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76b46ff5d4-rk57q" event={"ID":"f077e16d-a017-48e7-a15c-f8c1420ef50b","Type":"ContainerStarted","Data":"38b2e2b275b920976eee124441a2210782d441773cbd31e73f33347a5cbd786a"}
Jul 08 03:35:46.036399 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.036335    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76b46ff5d4-rk57q" podStartSLOduration=1.734555445 podStartE2EDuration="2.036311702s" totalImagesPullingTime="301.756257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:35:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:35:46.03049162 +0000 UTC m=+957.138837558" watchObservedRunningTime="2026-07-08 03:35:46.036311702 +0000 UTC m=+957.144657643"
Jul 08 03:35:46.063044 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.062967    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:35:46.063260 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.063222    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-66446d9d49-b48gv" podUID="157e92de-b877-4572-9fcc-7341f8747eb0" containerName="authorino" containerID="cri-o://6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9" gracePeriod=30
Jul 08 03:35:46.330525 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.330497    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:35:46.403103 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.403065    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert\") pod \"157e92de-b877-4572-9fcc-7341f8747eb0\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") "
Jul 08 03:35:46.403320 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.403164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gf4kr\" (UniqueName: \"kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr\") pod \"157e92de-b877-4572-9fcc-7341f8747eb0\" (UID: \"157e92de-b877-4572-9fcc-7341f8747eb0\") "
Jul 08 03:35:46.405295 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.405264    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr" (OuterVolumeSpecName: "kube-api-access-gf4kr") pod "157e92de-b877-4572-9fcc-7341f8747eb0" (UID: "157e92de-b877-4572-9fcc-7341f8747eb0"). InnerVolumeSpecName "kube-api-access-gf4kr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:35:46.414803 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.414764    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "157e92de-b877-4572-9fcc-7341f8747eb0" (UID: "157e92de-b877-4572-9fcc-7341f8747eb0"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:35:46.504204 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.504127    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/157e92de-b877-4572-9fcc-7341f8747eb0-tls-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:35:46.504204 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:46.504198    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gf4kr\" (UniqueName: \"kubernetes.io/projected/157e92de-b877-4572-9fcc-7341f8747eb0-kube-api-access-gf4kr\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:35:47.019791 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.019760    2563 generic.go:356] "Generic (PLEG): container finished" podID="157e92de-b877-4572-9fcc-7341f8747eb0" containerID="6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9" exitCode=0
Jul 08 03:35:47.020252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.019816    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-66446d9d49-b48gv"
Jul 08 03:35:47.020252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.019847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66446d9d49-b48gv" event={"ID":"157e92de-b877-4572-9fcc-7341f8747eb0","Type":"ContainerDied","Data":"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"}
Jul 08 03:35:47.020252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.019902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-66446d9d49-b48gv" event={"ID":"157e92de-b877-4572-9fcc-7341f8747eb0","Type":"ContainerDied","Data":"7af75f1d995d41f60e182f38f18d0de4050796e2fe40979fcb5f3a9381dafda2"}
Jul 08 03:35:47.020252 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.019928    2563 scope.go:117] "RemoveContainer" containerID="6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"
Jul 08 03:35:47.029609 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.029589    2563 scope.go:117] "RemoveContainer" containerID="6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"
Jul 08 03:35:47.030031 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:35:47.030008    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9\": container with ID starting with 6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9 not found: ID does not exist" containerID="6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"
Jul 08 03:35:47.030110 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.030043    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9"} err="failed to get container status \"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9\": rpc error: code = NotFound desc = could not find container \"6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9\": container with ID starting with 6cd75ac42c74d1a65f82a00271a92ab5eacabc3541565d4f5399d1a8092c07b9 not found: ID does not exist"
Jul 08 03:35:47.049105 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.049080    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:35:47.051560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.051540    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-66446d9d49-b48gv"]
Jul 08 03:35:47.389712 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:35:47.389670    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="157e92de-b877-4572-9fcc-7341f8747eb0" path="/var/lib/kubelet/pods/157e92de-b877-4572-9fcc-7341f8747eb0/volumes"
Jul 08 03:36:56.358933 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.358879    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:36:56.359637 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.359214    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-b857d6d59-bn2kv" podUID="e4051b7c-6aea-4449-b959-7c231ae2977a" containerName="manager" containerID="cri-o://6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5" gracePeriod=30
Jul 08 03:36:56.617541 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.617475    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:36:56.760289 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.760250    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert\") pod \"e4051b7c-6aea-4449-b959-7c231ae2977a\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") "
Jul 08 03:36:56.760460 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.760351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4thv\" (UniqueName: \"kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv\") pod \"e4051b7c-6aea-4449-b959-7c231ae2977a\" (UID: \"e4051b7c-6aea-4449-b959-7c231ae2977a\") "
Jul 08 03:36:56.762545 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.762522    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv" (OuterVolumeSpecName: "kube-api-access-q4thv") pod "e4051b7c-6aea-4449-b959-7c231ae2977a" (UID: "e4051b7c-6aea-4449-b959-7c231ae2977a"). InnerVolumeSpecName "kube-api-access-q4thv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:36:56.762660 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.762548    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e4051b7c-6aea-4449-b959-7c231ae2977a" (UID: "e4051b7c-6aea-4449-b959-7c231ae2977a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:36:56.861105 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.861071    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e4051b7c-6aea-4449-b959-7c231ae2977a-webhook-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:36:56.861105 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:56.861100    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4thv\" (UniqueName: \"kubernetes.io/projected/e4051b7c-6aea-4449-b959-7c231ae2977a-kube-api-access-q4thv\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:36:57.430698 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.430663    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4051b7c-6aea-4449-b959-7c231ae2977a" containerID="6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5" exitCode=0
Jul 08 03:36:57.431094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.430705    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-bn2kv"
Jul 08 03:36:57.431094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.430745    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-bn2kv" event={"ID":"e4051b7c-6aea-4449-b959-7c231ae2977a","Type":"ContainerDied","Data":"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"}
Jul 08 03:36:57.431094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.430801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-bn2kv" event={"ID":"e4051b7c-6aea-4449-b959-7c231ae2977a","Type":"ContainerDied","Data":"eb381dbe3a3a9a98d5047f359630fd45fcd732c5c5d17782a994121044785e1d"}
Jul 08 03:36:57.431094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.430828    2563 scope.go:117] "RemoveContainer" containerID="6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"
Jul 08 03:36:57.441116 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.441095    2563 scope.go:117] "RemoveContainer" containerID="6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"
Jul 08 03:36:57.441444 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:36:57.441424    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5\": container with ID starting with 6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5 not found: ID does not exist" containerID="6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"
Jul 08 03:36:57.441521 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.441452    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5"} err="failed to get container status \"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5\": rpc error: code = NotFound desc = could not find container \"6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5\": container with ID starting with 6290c99604a45406dc6aac4852eb82193a36d2537709d0e368f93327145649b5 not found: ID does not exist"
Jul 08 03:36:57.448923 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.448897    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:36:57.454494 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:57.454471    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-b857d6d59-bn2kv"]
Jul 08 03:36:58.243681 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.243650    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-b857d6d59-k95zk"]
Jul 08 03:36:58.244170 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244153    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="157e92de-b877-4572-9fcc-7341f8747eb0" containerName="authorino"
Jul 08 03:36:58.244170 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244172    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="157e92de-b877-4572-9fcc-7341f8747eb0" containerName="authorino"
Jul 08 03:36:58.244306 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244188    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4051b7c-6aea-4449-b959-7c231ae2977a" containerName="manager"
Jul 08 03:36:58.244306 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244194    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4051b7c-6aea-4449-b959-7c231ae2977a" containerName="manager"
Jul 08 03:36:58.244306 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244258    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4051b7c-6aea-4449-b959-7c231ae2977a" containerName="manager"
Jul 08 03:36:58.244306 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.244268    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="157e92de-b877-4572-9fcc-7341f8747eb0" containerName="authorino"
Jul 08 03:36:58.248638 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.248622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.251657 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.251636    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 03:36:58.251657 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.251643    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tvjh8\""
Jul 08 03:36:58.251949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.251932    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 03:36:58.259840 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.259815    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b857d6d59-k95zk"]
Jul 08 03:36:58.375556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.375519    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d8a6d-259b-4780-9571-3aa34bcfe8f5-webhook-cert\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.375556 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.375558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq58t\" (UniqueName: \"kubernetes.io/projected/632d8a6d-259b-4780-9571-3aa34bcfe8f5-kube-api-access-tq58t\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.476879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.476842    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d8a6d-259b-4780-9571-3aa34bcfe8f5-webhook-cert\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.477389 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.476888    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq58t\" (UniqueName: \"kubernetes.io/projected/632d8a6d-259b-4780-9571-3aa34bcfe8f5-kube-api-access-tq58t\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.479469 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.479442    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/632d8a6d-259b-4780-9571-3aa34bcfe8f5-webhook-cert\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.485804 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.485781    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq58t\" (UniqueName: \"kubernetes.io/projected/632d8a6d-259b-4780-9571-3aa34bcfe8f5-kube-api-access-tq58t\") pod \"maas-controller-b857d6d59-k95zk\" (UID: \"632d8a6d-259b-4780-9571-3aa34bcfe8f5\") " pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.559832 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.559799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:58.716499 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:58.716467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-b857d6d59-k95zk"]
Jul 08 03:36:58.718505 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:36:58.718477    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod632d8a6d_259b_4780_9571_3aa34bcfe8f5.slice/crio-37e4eb74f760042f92dfca8d5d49c07c0d04cc8091c793767d2f22c21cbecf38 WatchSource:0}: Error finding container 37e4eb74f760042f92dfca8d5d49c07c0d04cc8091c793767d2f22c21cbecf38: Status 404 returned error can't find the container with id 37e4eb74f760042f92dfca8d5d49c07c0d04cc8091c793767d2f22c21cbecf38
Jul 08 03:36:59.386203 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:59.386171    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4051b7c-6aea-4449-b959-7c231ae2977a" path="/var/lib/kubelet/pods/e4051b7c-6aea-4449-b959-7c231ae2977a/volumes"
Jul 08 03:36:59.445239 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:59.445202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-k95zk" event={"ID":"632d8a6d-259b-4780-9571-3aa34bcfe8f5","Type":"ContainerStarted","Data":"1f3a2c376657f3b6db994f0b90e2770580ad20405fddfeee0da47ec96a54e1ea"}
Jul 08 03:36:59.445422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:59.445262    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-b857d6d59-k95zk" event={"ID":"632d8a6d-259b-4780-9571-3aa34bcfe8f5","Type":"ContainerStarted","Data":"37e4eb74f760042f92dfca8d5d49c07c0d04cc8091c793767d2f22c21cbecf38"}
Jul 08 03:36:59.445422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:59.445352    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:36:59.464485 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:36:59.464425    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-b857d6d59-k95zk" podStartSLOduration=0.955333504 podStartE2EDuration="1.464406024s" totalImagesPullingTime="509.07252ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:36:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:36:59.462275455 +0000 UTC m=+1030.570621395" watchObservedRunningTime="2026-07-08 03:36:59.464406024 +0000 UTC m=+1030.572751965"
Jul 08 03:37:10.457879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:37:10.457844    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-b857d6d59-k95zk"
Jul 08 03:39:50.004942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:39:50.004836    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:39:50.008838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:39:50.005272    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:39:50.008960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:39:50.008946    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:39:50.009213 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:39:50.009197    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:41:43.687942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:43.687900    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:41:43.688517 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:43.688117    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" podUID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" containerName="manager" containerID="cri-o://305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c" gracePeriod=10
Jul 08 03:41:47.451158 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.451116    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:41:47.481492 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.481466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume\") pod \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") "
Jul 08 03:41:47.481677 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.481543    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hc87z\" (UniqueName: \"kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z\") pod \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\" (UID: \"0b986fb7-eb20-4d55-b74b-4e99d7df2282\") "
Jul 08 03:41:47.481876 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.481844    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b986fb7-eb20-4d55-b74b-4e99d7df2282" (UID: "0b986fb7-eb20-4d55-b74b-4e99d7df2282"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:41:47.483735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.483709    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z" (OuterVolumeSpecName: "kube-api-access-hc87z") pod "0b986fb7-eb20-4d55-b74b-4e99d7df2282" (UID: "0b986fb7-eb20-4d55-b74b-4e99d7df2282"). InnerVolumeSpecName "kube-api-access-hc87z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:41:47.582506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.582466    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b986fb7-eb20-4d55-b74b-4e99d7df2282-extensions-socket-volume\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:41:47.582506 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:47.582501    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hc87z\" (UniqueName: \"kubernetes.io/projected/0b986fb7-eb20-4d55-b74b-4e99d7df2282-kube-api-access-hc87z\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:41:48.146001 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.145963    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" containerID="305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c" exitCode=0
Jul 08 03:41:48.146217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.146022    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"
Jul 08 03:41:48.146217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.146058    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" event={"ID":"0b986fb7-eb20-4d55-b74b-4e99d7df2282","Type":"ContainerDied","Data":"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"}
Jul 08 03:41:48.146217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.146132    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls" event={"ID":"0b986fb7-eb20-4d55-b74b-4e99d7df2282","Type":"ContainerDied","Data":"2f35618d942be4837b1cd5ee2fd50580ffbff8e2438700a047b63cd9d78fbff6"}
Jul 08 03:41:48.146217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.146176    2563 scope.go:117] "RemoveContainer" containerID="305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"
Jul 08 03:41:48.157073 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.157055    2563 scope.go:117] "RemoveContainer" containerID="305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"
Jul 08 03:41:48.157307 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:41:48.157290    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c\": container with ID starting with 305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c not found: ID does not exist" containerID="305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"
Jul 08 03:41:48.157378 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.157316    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c"} err="failed to get container status \"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c\": rpc error: code = NotFound desc = could not find container \"305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c\": container with ID starting with 305c6a7eb1474e751c831d7d364080e47bd48afeaebf5b306afd22695fc9d59c not found: ID does not exist"
Jul 08 03:41:48.167355 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.167334    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:41:48.171780 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:48.171759    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9rsls"]
Jul 08 03:41:49.384907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:41:49.384873    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" path="/var/lib/kubelet/pods/0b986fb7-eb20-4d55-b74b-4e99d7df2282/volumes"
Jul 08 03:42:44.774178 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.774061    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"]
Jul 08 03:42:44.774816 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.774794    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" containerName="manager"
Jul 08 03:42:44.774907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.774842    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" containerName="manager"
Jul 08 03:42:44.775004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.774990    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b986fb7-eb20-4d55-b74b-4e99d7df2282" containerName="manager"
Jul 08 03:42:44.778342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.778319    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:44.781627 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.781607    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d2xbg\""
Jul 08 03:42:44.789249 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.789219    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"]
Jul 08 03:42:44.902388 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.902357    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:44.902560 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:44.902399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjrxn\" (UniqueName: \"kubernetes.io/projected/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-kube-api-access-jjrxn\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.003675 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.003640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.003861 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.003684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjrxn\" (UniqueName: \"kubernetes.io/projected/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-kube-api-access-jjrxn\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.004046 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.004026    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.012748 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.012719    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjrxn\" (UniqueName: \"kubernetes.io/projected/8d650c5b-7758-45a7-bbe6-6b2ebeabd132-kube-api-access-jjrxn\") pod \"kuadrant-operator-controller-manager-86878c769f-9z84k\" (UID: \"8d650c5b-7758-45a7-bbe6-6b2ebeabd132\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.090095 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.090057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:45.437239 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.437214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"]
Jul 08 03:42:45.439058 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:42:45.439031    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d650c5b_7758_45a7_bbe6_6b2ebeabd132.slice/crio-1b09d1bb842664064e6ed11205c9fb4fc6ecfe75b87cdee08769b870ccf520f3 WatchSource:0}: Error finding container 1b09d1bb842664064e6ed11205c9fb4fc6ecfe75b87cdee08769b870ccf520f3: Status 404 returned error can't find the container with id 1b09d1bb842664064e6ed11205c9fb4fc6ecfe75b87cdee08769b870ccf520f3
Jul 08 03:42:45.467867 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:45.467839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k" event={"ID":"8d650c5b-7758-45a7-bbe6-6b2ebeabd132","Type":"ContainerStarted","Data":"1b09d1bb842664064e6ed11205c9fb4fc6ecfe75b87cdee08769b870ccf520f3"}
Jul 08 03:42:46.475759 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:46.475723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k" event={"ID":"8d650c5b-7758-45a7-bbe6-6b2ebeabd132","Type":"ContainerStarted","Data":"c1c43fa35f643cb7b5af33d0847bf175f22330a7b692d1258a2eec7e01b558aa"}
Jul 08 03:42:46.476168 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:46.475845    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:42:46.496705 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:46.496643    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k" podStartSLOduration=2.496621471 podStartE2EDuration="2.496621471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:42:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:42:46.495710554 +0000 UTC m=+1377.604056491" watchObservedRunningTime="2026-07-08 03:42:46.496621471 +0000 UTC m=+1377.604967413"
Jul 08 03:42:57.484056 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:42:57.484028    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-9z84k"
Jul 08 03:44:50.044915 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:44:50.044885    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:44:50.048158 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:44:50.045937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:44:50.049111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:44:50.049094    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:44:50.049796 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:44:50.049780    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:45:00.137978 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.137942    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29724705-sjrf8"]
Jul 08 03:45:00.141829 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.141804    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:00.157428 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.157397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724705-sjrf8"]
Jul 08 03:45:00.267534 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.267503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk8wt\" (UniqueName: \"kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt\") pod \"maas-api-key-cleanup-29724705-sjrf8\" (UID: \"58723121-7b6d-4fbb-9ac8-eee93d38dfb9\") " pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:00.369062 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.369021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk8wt\" (UniqueName: \"kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt\") pod \"maas-api-key-cleanup-29724705-sjrf8\" (UID: \"58723121-7b6d-4fbb-9ac8-eee93d38dfb9\") " pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:00.377564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.377539    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk8wt\" (UniqueName: \"kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt\") pod \"maas-api-key-cleanup-29724705-sjrf8\" (UID: \"58723121-7b6d-4fbb-9ac8-eee93d38dfb9\") " pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:00.451856 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.451757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:00.798636 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.798610    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29724705-sjrf8"]
Jul 08 03:45:00.799652 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:45:00.799618    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58723121_7b6d_4fbb_9ac8_eee93d38dfb9.slice/crio-4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6 WatchSource:0}: Error finding container 4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6: Status 404 returned error can't find the container with id 4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6
Jul 08 03:45:00.801557 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:00.801542    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:45:01.267665 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:01.267579    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8" event={"ID":"58723121-7b6d-4fbb-9ac8-eee93d38dfb9","Type":"ContainerStarted","Data":"4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6"}
Jul 08 03:45:04.289180 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:04.289052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8" event={"ID":"58723121-7b6d-4fbb-9ac8-eee93d38dfb9","Type":"ContainerStarted","Data":"55b232c9f3a272c833cc97f1f7192196267261f2d086821530366f6ec4cbe0f3"}
Jul 08 03:45:04.303700 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:04.303640    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8" podStartSLOduration=1.494748408 podStartE2EDuration="4.303620706s" totalImagesPullingTime="2.808872298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:45:04.303443585 +0000 UTC m=+1515.411789527" watchObservedRunningTime="2026-07-08 03:45:04.303620706 +0000 UTC m=+1515.411967053"
Jul 08 03:45:05.298582 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:05.298549    2563 generic.go:356] "Generic (PLEG): container finished" podID="58723121-7b6d-4fbb-9ac8-eee93d38dfb9" containerID="55b232c9f3a272c833cc97f1f7192196267261f2d086821530366f6ec4cbe0f3" exitCode=0
Jul 08 03:45:05.298984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:05.298632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8" event={"ID":"58723121-7b6d-4fbb-9ac8-eee93d38dfb9","Type":"ContainerDied","Data":"55b232c9f3a272c833cc97f1f7192196267261f2d086821530366f6ec4cbe0f3"}
Jul 08 03:45:06.446317 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:06.446292    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:06.527041 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:06.527011    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mk8wt\" (UniqueName: \"kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt\") pod \"58723121-7b6d-4fbb-9ac8-eee93d38dfb9\" (UID: \"58723121-7b6d-4fbb-9ac8-eee93d38dfb9\") "
Jul 08 03:45:06.529210 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:06.529180    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt" (OuterVolumeSpecName: "kube-api-access-mk8wt") pod "58723121-7b6d-4fbb-9ac8-eee93d38dfb9" (UID: "58723121-7b6d-4fbb-9ac8-eee93d38dfb9"). InnerVolumeSpecName "kube-api-access-mk8wt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:45:06.628203 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:06.628083    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk8wt\" (UniqueName: \"kubernetes.io/projected/58723121-7b6d-4fbb-9ac8-eee93d38dfb9-kube-api-access-mk8wt\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:45:07.312564 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:07.312534    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8"
Jul 08 03:45:07.312781 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:07.312563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29724705-sjrf8" event={"ID":"58723121-7b6d-4fbb-9ac8-eee93d38dfb9","Type":"ContainerDied","Data":"4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6"}
Jul 08 03:45:07.312781 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:45:07.312611    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4ba3ef406e4ddf495496c6f5547e7794375662eaf1d343c2adc9f2fd054cadc6"
Jul 08 03:48:55.862073 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.862029    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:48:55.862635 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.862563    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58723121-7b6d-4fbb-9ac8-eee93d38dfb9" containerName="cleanup"
Jul 08 03:48:55.862635 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.862580    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="58723121-7b6d-4fbb-9ac8-eee93d38dfb9" containerName="cleanup"
Jul 08 03:48:55.862770 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.862686    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="58723121-7b6d-4fbb-9ac8-eee93d38dfb9" containerName="cleanup"
Jul 08 03:48:55.865431 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.865408    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:55.867411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.867389    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b547d5e7-serving-cert\""
Jul 08 03:48:55.877509 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.877482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:48:55.984854 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.984815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:55.985050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.984887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjbks\" (UniqueName: \"kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:55.985050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:55.984911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.085562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.085525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.085746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.085589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjbks\" (UniqueName: \"kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.085746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.085612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.086302 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.086275    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.088234 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.088212    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.098899 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.098874    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjbks\" (UniqueName: \"kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks\") pod \"maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") " pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.177939 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.177838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:56.335593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.335564    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:48:56.337032 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:48:56.337004    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f340184_9440_42a6_870c_9d53024f40d1.slice/crio-db689b5145c42b927583757d320ffecae9365675e4c9439d2d9525f591c96fd5 WatchSource:0}: Error finding container db689b5145c42b927583757d320ffecae9365675e4c9439d2d9525f591c96fd5: Status 404 returned error can't find the container with id db689b5145c42b927583757d320ffecae9365675e4c9439d2d9525f591c96fd5
Jul 08 03:48:56.696685 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:56.696655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" event={"ID":"2f340184-9440-42a6-870c-9d53024f40d1","Type":"ContainerStarted","Data":"db689b5145c42b927583757d320ffecae9365675e4c9439d2d9525f591c96fd5"}
Jul 08 03:48:57.706314 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:57.706280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" event={"ID":"2f340184-9440-42a6-870c-9d53024f40d1","Type":"ContainerStarted","Data":"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"}
Jul 08 03:48:57.706727 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:57.706405    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:48:57.728308 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:48:57.728261    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" podStartSLOduration=2.241267529 podStartE2EDuration="2.728245228s" totalImagesPullingTime="486.977699ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:48:57.72553301 +0000 UTC m=+1748.833878951" watchObservedRunningTime="2026-07-08 03:48:57.728245228 +0000 UTC m=+1748.836591172"
Jul 08 03:49:01.091486 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.091453    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:49:01.091977 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.091746    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" podUID="2f340184-9440-42a6-870c-9d53024f40d1" containerName="maas-api" containerID="cri-o://115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312" gracePeriod=30
Jul 08 03:49:01.098041 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.098017    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:49:01.352662 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.352598    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:49:01.440310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.440277    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls\") pod \"2f340184-9440-42a6-870c-9d53024f40d1\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") "
Jul 08 03:49:01.440310 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.440310    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjbks\" (UniqueName: \"kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks\") pod \"2f340184-9440-42a6-870c-9d53024f40d1\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") "
Jul 08 03:49:01.440522 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.440365    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca\") pod \"2f340184-9440-42a6-870c-9d53024f40d1\" (UID: \"2f340184-9440-42a6-870c-9d53024f40d1\") "
Jul 08 03:49:01.440826 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.440795    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f340184-9440-42a6-870c-9d53024f40d1" (UID: "2f340184-9440-42a6-870c-9d53024f40d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:49:01.442615 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.442588    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f340184-9440-42a6-870c-9d53024f40d1" (UID: "2f340184-9440-42a6-870c-9d53024f40d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:49:01.442727 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.442637    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks" (OuterVolumeSpecName: "kube-api-access-mjbks") pod "2f340184-9440-42a6-870c-9d53024f40d1" (UID: "2f340184-9440-42a6-870c-9d53024f40d1"). InnerVolumeSpecName "kube-api-access-mjbks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:49:01.541111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.541078    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f340184-9440-42a6-870c-9d53024f40d1-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:01.541111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.541106    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjbks\" (UniqueName: \"kubernetes.io/projected/2f340184-9440-42a6-870c-9d53024f40d1-kube-api-access-mjbks\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:01.541111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.541116    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f340184-9440-42a6-870c-9d53024f40d1-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:01.730592 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.730496    2563 generic.go:356] "Generic (PLEG): container finished" podID="2f340184-9440-42a6-870c-9d53024f40d1" containerID="115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312" exitCode=0
Jul 08 03:49:01.730592 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.730546    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"
Jul 08 03:49:01.730592 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.730568    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" event={"ID":"2f340184-9440-42a6-870c-9d53024f40d1","Type":"ContainerDied","Data":"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"}
Jul 08 03:49:01.730859 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.730632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl" event={"ID":"2f340184-9440-42a6-870c-9d53024f40d1","Type":"ContainerDied","Data":"db689b5145c42b927583757d320ffecae9365675e4c9439d2d9525f591c96fd5"}
Jul 08 03:49:01.730859 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.730659    2563 scope.go:117] "RemoveContainer" containerID="115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"
Jul 08 03:49:01.740869 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.740852    2563 scope.go:117] "RemoveContainer" containerID="115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"
Jul 08 03:49:01.741185 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:49:01.741133    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312\": container with ID starting with 115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312 not found: ID does not exist" containerID="115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"
Jul 08 03:49:01.741249 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.741194    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312"} err="failed to get container status \"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312\": rpc error: code = NotFound desc = could not find container \"115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312\": container with ID starting with 115fc3771c76411e0144f1ba6840f6b4e557a2aa64bb0ecaf25d02f03bbb4312 not found: ID does not exist"
Jul 08 03:49:01.753690 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.753660    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:49:01.755942 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:01.755920    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b547d5e7-78c6fdb9f-spdnl"]
Jul 08 03:49:03.386080 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:03.386040    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f340184-9440-42a6-870c-9d53024f40d1" path="/var/lib/kubelet/pods/2f340184-9440-42a6-870c-9d53024f40d1/volumes"
Jul 08 03:49:08.262588 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.262561    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:08.263213 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.263188    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f340184-9440-42a6-870c-9d53024f40d1" containerName="maas-api"
Jul 08 03:49:08.263213 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.263213    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f340184-9440-42a6-870c-9d53024f40d1" containerName="maas-api"
Jul 08 03:49:08.263410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.263306    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f340184-9440-42a6-870c-9d53024f40d1" containerName="maas-api"
Jul 08 03:49:08.267738 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.267718    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.270288 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.270266    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-a129cbbc-serving-cert\""
Jul 08 03:49:08.281843 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.281820    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:08.401223 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.401190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9x9v\" (UniqueName: \"kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.401416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.401241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.401416 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.401314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.501860 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.501825    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9x9v\" (UniqueName: \"kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.502065 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.501874    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.502065 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.501910    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.502671 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.502617    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.504528 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.504501    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.510294 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.510274    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9x9v\" (UniqueName: \"kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v\") pod \"maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") " pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.581951 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.581919    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:08.741566 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.741540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:08.743921 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:49:08.743890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf0be336_3943_4a45_96e8_efe4ab633932.slice/crio-8680b62e99f198f8d268150667e5d91bf97093cccfe660b2a255a6eb89567ed6 WatchSource:0}: Error finding container 8680b62e99f198f8d268150667e5d91bf97093cccfe660b2a255a6eb89567ed6: Status 404 returned error can't find the container with id 8680b62e99f198f8d268150667e5d91bf97093cccfe660b2a255a6eb89567ed6
Jul 08 03:49:08.773410 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:08.773366    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" event={"ID":"af0be336-3943-4a45-96e8-efe4ab633932","Type":"ContainerStarted","Data":"8680b62e99f198f8d268150667e5d91bf97093cccfe660b2a255a6eb89567ed6"}
Jul 08 03:49:09.782901 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:09.782856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" event={"ID":"af0be336-3943-4a45-96e8-efe4ab633932","Type":"ContainerStarted","Data":"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"}
Jul 08 03:49:09.783341 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:09.782941    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:09.799663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:09.799617    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" podStartSLOduration=1.408269285 podStartE2EDuration="1.799600673s" totalImagesPullingTime="391.331388ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:49:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:49:09.79844016 +0000 UTC m=+1760.906786119" watchObservedRunningTime="2026-07-08 03:49:09.799600673 +0000 UTC m=+1760.907946613"
Jul 08 03:49:13.474999 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.474970    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:13.475498 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.475169    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" podUID="af0be336-3943-4a45-96e8-efe4ab633932" containerName="maas-api" containerID="cri-o://8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e" gracePeriod=30
Jul 08 03:49:13.741470 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.741446    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:13.806435 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.806400    2563 generic.go:356] "Generic (PLEG): container finished" podID="af0be336-3943-4a45-96e8-efe4ab633932" containerID="8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e" exitCode=0
Jul 08 03:49:13.806650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.806450    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"
Jul 08 03:49:13.806650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.806488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" event={"ID":"af0be336-3943-4a45-96e8-efe4ab633932","Type":"ContainerDied","Data":"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"}
Jul 08 03:49:13.806650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.806583    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh" event={"ID":"af0be336-3943-4a45-96e8-efe4ab633932","Type":"ContainerDied","Data":"8680b62e99f198f8d268150667e5d91bf97093cccfe660b2a255a6eb89567ed6"}
Jul 08 03:49:13.806650 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.806617    2563 scope.go:117] "RemoveContainer" containerID="8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"
Jul 08 03:49:13.815813 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.815796    2563 scope.go:117] "RemoveContainer" containerID="8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"
Jul 08 03:49:13.816073 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:49:13.816056    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e\": container with ID starting with 8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e not found: ID does not exist" containerID="8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"
Jul 08 03:49:13.816118 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.816082    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e"} err="failed to get container status \"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e\": rpc error: code = NotFound desc = could not find container \"8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e\": container with ID starting with 8cbf675a7b21719d1215124d3a8164b276249ea01d77ea661f4a448b6c73dd2e not found: ID does not exist"
Jul 08 03:49:13.854668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.854641    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9x9v\" (UniqueName: \"kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v\") pod \"af0be336-3943-4a45-96e8-efe4ab633932\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") "
Jul 08 03:49:13.854806 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.854685    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls\") pod \"af0be336-3943-4a45-96e8-efe4ab633932\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") "
Jul 08 03:49:13.854806 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.854711    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca\") pod \"af0be336-3943-4a45-96e8-efe4ab633932\" (UID: \"af0be336-3943-4a45-96e8-efe4ab633932\") "
Jul 08 03:49:13.855123 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.855097    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af0be336-3943-4a45-96e8-efe4ab633932" (UID: "af0be336-3943-4a45-96e8-efe4ab633932"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:49:13.856890 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.856860    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v" (OuterVolumeSpecName: "kube-api-access-b9x9v") pod "af0be336-3943-4a45-96e8-efe4ab633932" (UID: "af0be336-3943-4a45-96e8-efe4ab633932"). InnerVolumeSpecName "kube-api-access-b9x9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:49:13.856984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.856953    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af0be336-3943-4a45-96e8-efe4ab633932" (UID: "af0be336-3943-4a45-96e8-efe4ab633932"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:49:13.956342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.956295    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af0be336-3943-4a45-96e8-efe4ab633932-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:13.956342 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.956340    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9x9v\" (UniqueName: \"kubernetes.io/projected/af0be336-3943-4a45-96e8-efe4ab633932-kube-api-access-b9x9v\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:13.956562 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:13.956356    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af0be336-3943-4a45-96e8-efe4ab633932-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:14.128578 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:14.128542    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:14.134538 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:14.134513    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-a129cbbc-6c9498c969-7p6hh"]
Jul 08 03:49:15.386996 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:15.386961    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0be336-3943-4a45-96e8-efe4ab633932" path="/var/lib/kubelet/pods/af0be336-3943-4a45-96e8-efe4ab633932/volumes"
Jul 08 03:49:26.867113 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.867081    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:26.867689 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.867516    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0be336-3943-4a45-96e8-efe4ab633932" containerName="maas-api"
Jul 08 03:49:26.867689 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.867529    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0be336-3943-4a45-96e8-efe4ab633932" containerName="maas-api"
Jul 08 03:49:26.867689 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.867601    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="af0be336-3943-4a45-96e8-efe4ab633932" containerName="maas-api"
Jul 08 03:49:26.870872 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.870840    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:26.873409 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.873390    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-afef641e-serving-cert\""
Jul 08 03:49:26.879658 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.879635    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:26.976663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.976620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:26.976940 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.976920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:26.977169 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:26.977125    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzpw8\" (UniqueName: \"kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.078593 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.078552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.078798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.078611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.078798 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.078671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzpw8\" (UniqueName: \"kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.078944 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:49:27.078833    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-afef641e-serving-cert: secret "maas-api-e2e-derive-afef641e-serving-cert" not found
Jul 08 03:49:27.078944 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:49:27.078922    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls podName:ef6a8fc7-1b91-4d41-b340-2735580a06f0 nodeName:}" failed. No retries permitted until 2026-07-08 03:49:27.578898666 +0000 UTC m=+1778.687244773 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls") pod "maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" (UID: "ef6a8fc7-1b91-4d41-b340-2735580a06f0") : secret "maas-api-e2e-derive-afef641e-serving-cert" not found
Jul 08 03:49:27.079501 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.079473    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.088643 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.088614    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzpw8\" (UniqueName: \"kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.584123 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.584092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.586716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.586689    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") pod \"maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") " pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.785666 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.785617    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:27.940422 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:27.940388    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:27.944917 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:49:27.944883    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6a8fc7_1b91_4d41_b340_2735580a06f0.slice/crio-67dc34c60e5210584e0f854502ebd5b1805fdf5970c52573b3bfea327ea2af59 WatchSource:0}: Error finding container 67dc34c60e5210584e0f854502ebd5b1805fdf5970c52573b3bfea327ea2af59: Status 404 returned error can't find the container with id 67dc34c60e5210584e0f854502ebd5b1805fdf5970c52573b3bfea327ea2af59
Jul 08 03:49:28.902841 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:28.902797    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" event={"ID":"ef6a8fc7-1b91-4d41-b340-2735580a06f0","Type":"ContainerStarted","Data":"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"}
Jul 08 03:49:28.903062 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:28.902887    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:28.903062 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:28.902919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" event={"ID":"ef6a8fc7-1b91-4d41-b340-2735580a06f0","Type":"ContainerStarted","Data":"67dc34c60e5210584e0f854502ebd5b1805fdf5970c52573b3bfea327ea2af59"}
Jul 08 03:49:28.922165 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:28.922091    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" podStartSLOduration=2.62183281 podStartE2EDuration="2.922073714s" totalImagesPullingTime="300.240904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:49:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:49:28.921637971 +0000 UTC m=+1780.029983911" watchObservedRunningTime="2026-07-08 03:49:28.922073714 +0000 UTC m=+1780.030419653"
Jul 08 03:49:31.505569 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.505539    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:31.506000 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.505770    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" podUID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" containerName="maas-api" containerID="cri-o://3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc" gracePeriod=30
Jul 08 03:49:31.753931 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.753904    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:31.826298 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.826266    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca\") pod \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") "
Jul 08 03:49:31.826500 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.826310    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") pod \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") "
Jul 08 03:49:31.826500 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.826396    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rzpw8\" (UniqueName: \"kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8\") pod \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\" (UID: \"ef6a8fc7-1b91-4d41-b340-2735580a06f0\") "
Jul 08 03:49:31.826732 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.826708    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef6a8fc7-1b91-4d41-b340-2735580a06f0" (UID: "ef6a8fc7-1b91-4d41-b340-2735580a06f0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:49:31.828649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.828613    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8" (OuterVolumeSpecName: "kube-api-access-rzpw8") pod "ef6a8fc7-1b91-4d41-b340-2735580a06f0" (UID: "ef6a8fc7-1b91-4d41-b340-2735580a06f0"). InnerVolumeSpecName "kube-api-access-rzpw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:49:31.828760 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.828698    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef6a8fc7-1b91-4d41-b340-2735580a06f0" (UID: "ef6a8fc7-1b91-4d41-b340-2735580a06f0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:49:31.921487 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.921453    2563 generic.go:356] "Generic (PLEG): container finished" podID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" containerID="3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc" exitCode=0
Jul 08 03:49:31.921669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.921502    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"
Jul 08 03:49:31.921669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.921501    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" event={"ID":"ef6a8fc7-1b91-4d41-b340-2735580a06f0","Type":"ContainerDied","Data":"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"}
Jul 08 03:49:31.921669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.921631    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc" event={"ID":"ef6a8fc7-1b91-4d41-b340-2735580a06f0","Type":"ContainerDied","Data":"67dc34c60e5210584e0f854502ebd5b1805fdf5970c52573b3bfea327ea2af59"}
Jul 08 03:49:31.921669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.921660    2563 scope.go:117] "RemoveContainer" containerID="3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"
Jul 08 03:49:31.927369 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.927337    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzpw8\" (UniqueName: \"kubernetes.io/projected/ef6a8fc7-1b91-4d41-b340-2735580a06f0-kube-api-access-rzpw8\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:31.927369 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.927362    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6a8fc7-1b91-4d41-b340-2735580a06f0-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:31.927521 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.927376    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef6a8fc7-1b91-4d41-b340-2735580a06f0-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:49:31.934812 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.934695    2563 scope.go:117] "RemoveContainer" containerID="3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"
Jul 08 03:49:31.935766 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:49:31.935741    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc\": container with ID starting with 3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc not found: ID does not exist" containerID="3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"
Jul 08 03:49:31.935868 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.935773    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc"} err="failed to get container status \"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc\": rpc error: code = NotFound desc = could not find container \"3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc\": container with ID starting with 3914df37515e25f92d3e68e5dc8900753d0729fee179fbd945497852a6c6a5cc not found: ID does not exist"
Jul 08 03:49:31.945496 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.945462    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:31.947863 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:31.947842    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-afef641e-6dcb577d77-cxkrc"]
Jul 08 03:49:33.386493 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:33.386456    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" path="/var/lib/kubelet/pods/ef6a8fc7-1b91-4d41-b340-2735580a06f0/volumes"
Jul 08 03:49:50.085790 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:50.085688    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:49:50.099181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:50.088985    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:49:50.099181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:50.090252    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:49:50.100326 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:49:50.100307    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:51:15.293683 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.293588    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:15.294274 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.294255    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" containerName="maas-api"
Jul 08 03:51:15.294345 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.294278    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" containerName="maas-api"
Jul 08 03:51:15.294438 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.294423    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef6a8fc7-1b91-4d41-b340-2735580a06f0" containerName="maas-api"
Jul 08 03:51:15.298183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.298161    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.303672 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.303642    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-fcd8d42d-serving-cert\""
Jul 08 03:51:15.304994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.304970    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:15.316383 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.316352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw6bm\" (UniqueName: \"kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.316512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.316399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.316512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.316419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.416890 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.416851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nw6bm\" (UniqueName: \"kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.417101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.416907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.417101 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.416931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.417101 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:51:15.417059    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-fcd8d42d-serving-cert: secret "maas-api-e2e-mt-fcd8d42d-serving-cert" not found
Jul 08 03:51:15.417317 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:51:15.417131    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls podName:d21d3dbf-c942-4776-bc12-e39765cfd886 nodeName:}" failed. No retries permitted until 2026-07-08 03:51:15.917110881 +0000 UTC m=+1887.025456820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls") pod "maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" (UID: "d21d3dbf-c942-4776-bc12-e39765cfd886") : secret "maas-api-e2e-mt-fcd8d42d-serving-cert" not found
Jul 08 03:51:15.417639 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.417614    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.430112 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.430071    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw6bm\" (UniqueName: \"kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.921649 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.921611    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:15.924292 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:15.924271    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") pod \"maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") " pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:16.213843 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:16.213757    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:16.362437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:16.362415    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:16.364765 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:51:16.364738    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd21d3dbf_c942_4776_bc12_e39765cfd886.slice/crio-b5ad3cb7beccc1568ace7eebdef62e8a52166203fc84a07ddb9e3be7823d0b0e WatchSource:0}: Error finding container b5ad3cb7beccc1568ace7eebdef62e8a52166203fc84a07ddb9e3be7823d0b0e: Status 404 returned error can't find the container with id b5ad3cb7beccc1568ace7eebdef62e8a52166203fc84a07ddb9e3be7823d0b0e
Jul 08 03:51:16.366129 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:16.366107    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:51:16.552691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:16.552658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" event={"ID":"d21d3dbf-c942-4776-bc12-e39765cfd886","Type":"ContainerStarted","Data":"b5ad3cb7beccc1568ace7eebdef62e8a52166203fc84a07ddb9e3be7823d0b0e"}
Jul 08 03:51:17.560521 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:17.560488    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" event={"ID":"d21d3dbf-c942-4776-bc12-e39765cfd886","Type":"ContainerStarted","Data":"42c072954cc11a95f351f392ca2dd4552cbb49837ba45b23ef1246877d66b74f"}
Jul 08 03:51:17.560938 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:17.560551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:17.579871 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:17.579810    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" podStartSLOduration=2.141698872 podStartE2EDuration="2.57979018s" totalImagesPullingTime="438.091308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:51:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:51:17.576269861 +0000 UTC m=+1888.684615804" watchObservedRunningTime="2026-07-08 03:51:17.57979018 +0000 UTC m=+1888.688136120"
Jul 08 03:51:23.572270 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:23.572240    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:27.436234 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.436197    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:27.436669 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.436449    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" podUID="d21d3dbf-c942-4776-bc12-e39765cfd886" containerName="maas-api" containerID="cri-o://42c072954cc11a95f351f392ca2dd4552cbb49837ba45b23ef1246877d66b74f" gracePeriod=30
Jul 08 03:51:27.624735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.624643    2563 generic.go:356] "Generic (PLEG): container finished" podID="d21d3dbf-c942-4776-bc12-e39765cfd886" containerID="42c072954cc11a95f351f392ca2dd4552cbb49837ba45b23ef1246877d66b74f" exitCode=0
Jul 08 03:51:27.624735 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.624714    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" event={"ID":"d21d3dbf-c942-4776-bc12-e39765cfd886","Type":"ContainerDied","Data":"42c072954cc11a95f351f392ca2dd4552cbb49837ba45b23ef1246877d66b74f"}
Jul 08 03:51:27.703852 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.703794    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:27.740614 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.740581    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw6bm\" (UniqueName: \"kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm\") pod \"d21d3dbf-c942-4776-bc12-e39765cfd886\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") "
Jul 08 03:51:27.740793 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.740639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca\") pod \"d21d3dbf-c942-4776-bc12-e39765cfd886\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") "
Jul 08 03:51:27.740793 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.740657    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") pod \"d21d3dbf-c942-4776-bc12-e39765cfd886\" (UID: \"d21d3dbf-c942-4776-bc12-e39765cfd886\") "
Jul 08 03:51:27.741084 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.741046    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d21d3dbf-c942-4776-bc12-e39765cfd886" (UID: "d21d3dbf-c942-4776-bc12-e39765cfd886"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:51:27.743026 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.743000    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm" (OuterVolumeSpecName: "kube-api-access-nw6bm") pod "d21d3dbf-c942-4776-bc12-e39765cfd886" (UID: "d21d3dbf-c942-4776-bc12-e39765cfd886"). InnerVolumeSpecName "kube-api-access-nw6bm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:51:27.743384 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.743360    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d21d3dbf-c942-4776-bc12-e39765cfd886" (UID: "d21d3dbf-c942-4776-bc12-e39765cfd886"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:51:27.841982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.841937    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw6bm\" (UniqueName: \"kubernetes.io/projected/d21d3dbf-c942-4776-bc12-e39765cfd886-kube-api-access-nw6bm\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:51:27.841982 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.841984    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d21d3dbf-c942-4776-bc12-e39765cfd886-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:51:27.842223 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:27.842001    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d21d3dbf-c942-4776-bc12-e39765cfd886-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:51:28.631671 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:28.631637    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"
Jul 08 03:51:28.631671 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:28.631666    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd" event={"ID":"d21d3dbf-c942-4776-bc12-e39765cfd886","Type":"ContainerDied","Data":"b5ad3cb7beccc1568ace7eebdef62e8a52166203fc84a07ddb9e3be7823d0b0e"}
Jul 08 03:51:28.632125 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:28.631722    2563 scope.go:117] "RemoveContainer" containerID="42c072954cc11a95f351f392ca2dd4552cbb49837ba45b23ef1246877d66b74f"
Jul 08 03:51:28.652603 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:28.652577    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:28.658657 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:28.658633    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-fcd8d42d-bbd8fbb8d-t5gsd"]
Jul 08 03:51:29.387236 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:29.387191    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d21d3dbf-c942-4776-bc12-e39765cfd886" path="/var/lib/kubelet/pods/d21d3dbf-c942-4776-bc12-e39765cfd886/volumes"
Jul 08 03:51:56.426032 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.425992    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:51:56.426694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.426676    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d21d3dbf-c942-4776-bc12-e39765cfd886" containerName="maas-api"
Jul 08 03:51:56.426746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.426698    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d21d3dbf-c942-4776-bc12-e39765cfd886" containerName="maas-api"
Jul 08 03:51:56.426821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.426810    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d21d3dbf-c942-4776-bc12-e39765cfd886" containerName="maas-api"
Jul 08 03:51:56.430947 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.430924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.433303 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.433278    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-11ba793b-serving-cert\""
Jul 08 03:51:56.438283 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.438253    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:51:56.518251 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.518217    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.518437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.518336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mkv\" (UniqueName: \"kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.518437 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.518370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.618901 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.618863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mkv\" (UniqueName: \"kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.618901 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.618907    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.619238 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.618943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.619238 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:51:56.619041    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-11ba793b-serving-cert: secret "maas-api-e2e-mt-11ba793b-serving-cert" not found
Jul 08 03:51:56.619238 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:51:56.619103    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls podName:df8faffe-b290-489c-93d0-036694bbc14e nodeName:}" failed. No retries permitted until 2026-07-08 03:51:57.119086461 +0000 UTC m=+1928.227432384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls") pod "maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" (UID: "df8faffe-b290-489c-93d0-036694bbc14e") : secret "maas-api-e2e-mt-11ba793b-serving-cert" not found
Jul 08 03:51:56.619746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.619722    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:56.630347 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:56.630318    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mkv\" (UniqueName: \"kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:57.123621 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:57.123579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:57.126262 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:57.126241    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") pod \"maas-api-e2e-mt-11ba793b-747db5c569-6r4j5\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") " pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:57.345415 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:57.345378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:57.499882 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:57.499842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:51:57.501365 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:51:57.501335    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf8faffe_b290_489c_93d0_036694bbc14e.slice/crio-ad9879c381bb4312f4a159a9ccd093cac819ff8d47543ed9e3d374282c8162e2 WatchSource:0}: Error finding container ad9879c381bb4312f4a159a9ccd093cac819ff8d47543ed9e3d374282c8162e2: Status 404 returned error can't find the container with id ad9879c381bb4312f4a159a9ccd093cac819ff8d47543ed9e3d374282c8162e2
Jul 08 03:51:57.803133 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:57.803092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" event={"ID":"df8faffe-b290-489c-93d0-036694bbc14e","Type":"ContainerStarted","Data":"ad9879c381bb4312f4a159a9ccd093cac819ff8d47543ed9e3d374282c8162e2"}
Jul 08 03:51:58.811090 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:58.811057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" event={"ID":"df8faffe-b290-489c-93d0-036694bbc14e","Type":"ContainerStarted","Data":"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"}
Jul 08 03:51:58.811546 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:58.811208    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:51:58.829463 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:51:58.829393    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" podStartSLOduration=2.458116994 podStartE2EDuration="2.829373335s" totalImagesPullingTime="371.256341ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:51:58.828668212 +0000 UTC m=+1929.937014153" watchObservedRunningTime="2026-07-08 03:51:58.829373335 +0000 UTC m=+1929.937719275"
Jul 08 03:52:04.824557 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:04.824526    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:52:49.457979 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.457898    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:52:49.458504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.458197    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" podUID="df8faffe-b290-489c-93d0-036694bbc14e" containerName="maas-api" containerID="cri-o://c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053" gracePeriod=30
Jul 08 03:52:49.714983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.714918    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:52:49.793375 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.793344    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca\") pod \"df8faffe-b290-489c-93d0-036694bbc14e\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") "
Jul 08 03:52:49.793549 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.793400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") pod \"df8faffe-b290-489c-93d0-036694bbc14e\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") "
Jul 08 03:52:49.793549 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.793494    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4mkv\" (UniqueName: \"kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv\") pod \"df8faffe-b290-489c-93d0-036694bbc14e\" (UID: \"df8faffe-b290-489c-93d0-036694bbc14e\") "
Jul 08 03:52:49.793740 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.793716    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df8faffe-b290-489c-93d0-036694bbc14e" (UID: "df8faffe-b290-489c-93d0-036694bbc14e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:52:49.793824 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.793793    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df8faffe-b290-489c-93d0-036694bbc14e-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:52:49.795641 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.795616    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv" (OuterVolumeSpecName: "kube-api-access-n4mkv") pod "df8faffe-b290-489c-93d0-036694bbc14e" (UID: "df8faffe-b290-489c-93d0-036694bbc14e"). InnerVolumeSpecName "kube-api-access-n4mkv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:52:49.795641 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.795628    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df8faffe-b290-489c-93d0-036694bbc14e" (UID: "df8faffe-b290-489c-93d0-036694bbc14e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:52:49.894411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.894364    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4mkv\" (UniqueName: \"kubernetes.io/projected/df8faffe-b290-489c-93d0-036694bbc14e-kube-api-access-n4mkv\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:52:49.894411 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:49.894400    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df8faffe-b290-489c-93d0-036694bbc14e-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:52:50.121584 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.121548    2563 generic.go:356] "Generic (PLEG): container finished" podID="df8faffe-b290-489c-93d0-036694bbc14e" containerID="c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053" exitCode=0
Jul 08 03:52:50.121787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.121595    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"
Jul 08 03:52:50.121787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.121643    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" event={"ID":"df8faffe-b290-489c-93d0-036694bbc14e","Type":"ContainerDied","Data":"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"}
Jul 08 03:52:50.121787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.121698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5" event={"ID":"df8faffe-b290-489c-93d0-036694bbc14e","Type":"ContainerDied","Data":"ad9879c381bb4312f4a159a9ccd093cac819ff8d47543ed9e3d374282c8162e2"}
Jul 08 03:52:50.121787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.121723    2563 scope.go:117] "RemoveContainer" containerID="c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"
Jul 08 03:52:50.132249 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.132234    2563 scope.go:117] "RemoveContainer" containerID="c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"
Jul 08 03:52:50.132485 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:52:50.132467    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053\": container with ID starting with c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053 not found: ID does not exist" containerID="c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"
Jul 08 03:52:50.132528 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.132495    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053"} err="failed to get container status \"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053\": rpc error: code = NotFound desc = could not find container \"c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053\": container with ID starting with c6b9bd79994224c6c667ef160c0a503a4250f607fe5f159aa60c9b7b4151d053 not found: ID does not exist"
Jul 08 03:52:50.144109 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.144082    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:52:50.147828 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:50.147806    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-11ba793b-747db5c569-6r4j5"]
Jul 08 03:52:51.386559 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:52:51.386524    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df8faffe-b290-489c-93d0-036694bbc14e" path="/var/lib/kubelet/pods/df8faffe-b290-489c-93d0-036694bbc14e/volumes"
Jul 08 03:54:40.153254 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.153210    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:54:40.153857 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.153835    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df8faffe-b290-489c-93d0-036694bbc14e" containerName="maas-api"
Jul 08 03:54:40.153949 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.153861    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="df8faffe-b290-489c-93d0-036694bbc14e" containerName="maas-api"
Jul 08 03:54:40.154011 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.153991    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="df8faffe-b290-489c-93d0-036694bbc14e" containerName="maas-api"
Jul 08 03:54:40.157787 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.157764    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.160706 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.160677    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-78a872-serving-cert\""
Jul 08 03:54:40.167092 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.167038    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:54:40.242507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.242467    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94vx9\" (UniqueName: \"kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.242686 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.242574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.242686 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.242651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.343550 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.343509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94vx9\" (UniqueName: \"kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.343765 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.343589    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.343765 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.343631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.344358 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.344333    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.346288 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.346270    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.352469 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.352447    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-94vx9\" (UniqueName: \"kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9\") pod \"maas-api-e2e-inf-a-78a872-5c756b5884-72h4t\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") " pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.472571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.472481    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:40.621305 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.621275    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:54:40.623725 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:54:40.623669    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7657b28_a2f2_4a53_9a4c_8bee91d5a05b.slice/crio-669e5cb2f6cca4a77018b723596842ed6ab0a0a58ab7fd9ee8d8d796d9e5a95f WatchSource:0}: Error finding container 669e5cb2f6cca4a77018b723596842ed6ab0a0a58ab7fd9ee8d8d796d9e5a95f: Status 404 returned error can't find the container with id 669e5cb2f6cca4a77018b723596842ed6ab0a0a58ab7fd9ee8d8d796d9e5a95f
Jul 08 03:54:40.762414 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:40.762326    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" event={"ID":"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b","Type":"ContainerStarted","Data":"669e5cb2f6cca4a77018b723596842ed6ab0a0a58ab7fd9ee8d8d796d9e5a95f"}
Jul 08 03:54:41.770192 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:41.770154    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" event={"ID":"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b","Type":"ContainerStarted","Data":"e7e8479b2e1e06b7a1ecec40dac0368facf5d487181689d522097ad0bd137a35"}
Jul 08 03:54:41.770576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:41.770284    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:41.787879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:41.787824    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" podStartSLOduration=1.15729106 podStartE2EDuration="1.78780686s" totalImagesPullingTime="630.5158ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:54:41.784910777 +0000 UTC m=+2092.893256717" watchObservedRunningTime="2026-07-08 03:54:41.78780686 +0000 UTC m=+2092.896152800"
Jul 08 03:54:47.784752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:47.784720    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:54:48.155606 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.155558    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:54:48.159736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.159712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.162399 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.162374    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-18addd-serving-cert\""
Jul 08 03:54:48.168504 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.168479    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:54:48.218936 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.218903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w48b5\" (UniqueName: \"kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.218936 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.218942    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.219170 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.218979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.319922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.319889    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w48b5\" (UniqueName: \"kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.320156 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.319927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.320156 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.319968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.320156 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:54:48.320086    2563 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-18addd-serving-cert: secret "maas-api-e2e-inf-b-18addd-serving-cert" not found
Jul 08 03:54:48.320340 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:54:48.320175    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls podName:71c90211-684a-4878-87f0-ecd06ee6cde5 nodeName:}" failed. No retries permitted until 2026-07-08 03:54:48.82015808 +0000 UTC m=+2099.928503999 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls") pod "maas-api-e2e-inf-b-18addd-856555dd5b-95g55" (UID: "71c90211-684a-4878-87f0-ecd06ee6cde5") : secret "maas-api-e2e-inf-b-18addd-serving-cert" not found
Jul 08 03:54:48.320558 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.320540    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.328361 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.328340    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w48b5\" (UniqueName: \"kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.824038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.824007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:48.826729 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:48.826704    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") pod \"maas-api-e2e-inf-b-18addd-856555dd5b-95g55\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") " pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:49.074984 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.074894    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:49.221763 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.221733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:54:49.223330 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:54:49.223304    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71c90211_684a_4878_87f0_ecd06ee6cde5.slice/crio-d9054da7596d9dba87c75be512e460a8513052b7efdcb7db645e9e43be187a27 WatchSource:0}: Error finding container d9054da7596d9dba87c75be512e460a8513052b7efdcb7db645e9e43be187a27: Status 404 returned error can't find the container with id d9054da7596d9dba87c75be512e460a8513052b7efdcb7db645e9e43be187a27
Jul 08 03:54:49.821576 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.821541    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" event={"ID":"71c90211-684a-4878-87f0-ecd06ee6cde5","Type":"ContainerStarted","Data":"047c9984914a32016aa9817e306e2365f74d710c76363aab844b200f358f675a"}
Jul 08 03:54:49.821767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.821605    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" event={"ID":"71c90211-684a-4878-87f0-ecd06ee6cde5","Type":"ContainerStarted","Data":"d9054da7596d9dba87c75be512e460a8513052b7efdcb7db645e9e43be187a27"}
Jul 08 03:54:49.821767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.821637    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:54:49.840465 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:49.840418    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" podStartSLOduration=1.538079813 podStartE2EDuration="1.840401009s" totalImagesPullingTime="302.321196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:54:49.837794565 +0000 UTC m=+2100.946140506" watchObservedRunningTime="2026-07-08 03:54:49.840401009 +0000 UTC m=+2100.948746948"
Jul 08 03:54:50.135809 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:50.135628    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:54:50.154767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:50.140851    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:54:50.154767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:50.141354    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:54:50.154767 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:50.145852    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:54:55.836759 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:54:55.836718    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:55:33.235929 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.235893    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:55:33.239627 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.239609    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.241688 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.241661    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-18addd\"/\"test-model-18addd-kserve-self-signed-certs\""
Jul 08 03:55:33.241688 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.241679    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-18addd\"/\"kube-root-ca.crt\""
Jul 08 03:55:33.241899 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.241661    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-18addd\"/\"openshift-service-ca.crt\""
Jul 08 03:55:33.242217 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.242202    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-18addd\"/\"default-dockercfg-pgk7x\""
Jul 08 03:55:33.249879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.249856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:55:33.356307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356276    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.356488 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.356488 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356336    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.356488 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.356595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356507    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.356595 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.356531    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fd9j5\" (UniqueName: \"kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457631 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457593    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457631 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457629    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457650    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fd9j5\" (UniqueName: \"kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457730    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.457884 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.457784    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.458186 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.458162    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.458278 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.458253    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.458339 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.458303    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.460224 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.460208    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.460324 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.460309    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.465789 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.465760    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fd9j5\" (UniqueName: \"kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5\") pod \"test-model-18addd-kserve-8d8f76c65-8vccw\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") " pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.550194 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.550126    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:33.693981 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:55:33.693951    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5999ba46_6f87_4379_bdd6_9661701eadaa.slice/crio-e1ca312e55f343c7ac220222774f929cdeb50ce33be780e63c2a94ca5a89798f WatchSource:0}: Error finding container e1ca312e55f343c7ac220222774f929cdeb50ce33be780e63c2a94ca5a89798f: Status 404 returned error can't find the container with id e1ca312e55f343c7ac220222774f929cdeb50ce33be780e63c2a94ca5a89798f
Jul 08 03:55:33.694129 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:33.694001    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:55:34.085833 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:34.085794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" event={"ID":"5999ba46-6f87-4379-bdd6-9661701eadaa","Type":"ContainerStarted","Data":"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"}
Jul 08 03:55:34.085833 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:34.085838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" event={"ID":"5999ba46-6f87-4379-bdd6-9661701eadaa","Type":"ContainerStarted","Data":"e1ca312e55f343c7ac220222774f929cdeb50ce33be780e63c2a94ca5a89798f"}
Jul 08 03:55:34.104474 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:34.104411    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" podStartSLOduration=1.104387745 podStartE2EDuration="1.104387745s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:55:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:55:34.102378883 +0000 UTC m=+2145.210724824" watchObservedRunningTime="2026-07-08 03:55:34.104387745 +0000 UTC m=+2145.212733688"
Jul 08 03:55:43.550704 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:43.550622    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:43.550704 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:43.550679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:43.563438 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:43.563411    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:55:44.156875 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:55:44.156849    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:56:23.343821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:23.343786    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:56:24.250487 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.250455    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:56:24.250728 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.250695    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" podUID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" containerName="maas-api" containerID="cri-o://e7e8479b2e1e06b7a1ecec40dac0368facf5d487181689d522097ad0bd137a35" gracePeriod=30
Jul 08 03:56:24.384168 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.384033    2563 generic.go:356] "Generic (PLEG): container finished" podID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" containerID="e7e8479b2e1e06b7a1ecec40dac0368facf5d487181689d522097ad0bd137a35" exitCode=0
Jul 08 03:56:24.384168 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.384086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" event={"ID":"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b","Type":"ContainerDied","Data":"e7e8479b2e1e06b7a1ecec40dac0368facf5d487181689d522097ad0bd137a35"}
Jul 08 03:56:24.514420 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.514357    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:56:24.543224 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.543192    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94vx9\" (UniqueName: \"kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9\") pod \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") "
Jul 08 03:56:24.543408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.543314    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls\") pod \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") "
Jul 08 03:56:24.543480 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.543413    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca\") pod \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\" (UID: \"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b\") "
Jul 08 03:56:24.543792 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.543768    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" (UID: "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:56:24.545599 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.545568    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" (UID: "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:56:24.545691 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.545601    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9" (OuterVolumeSpecName: "kube-api-access-94vx9") pod "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" (UID: "f7657b28-a2f2-4a53-9a4c-8bee91d5a05b"). InnerVolumeSpecName "kube-api-access-94vx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:56:24.644580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.644546    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:24.644580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.644573    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:24.644580 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:24.644586    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94vx9\" (UniqueName: \"kubernetes.io/projected/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b-kube-api-access-94vx9\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:25.394085 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:25.394050    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t" event={"ID":"f7657b28-a2f2-4a53-9a4c-8bee91d5a05b","Type":"ContainerDied","Data":"669e5cb2f6cca4a77018b723596842ed6ab0a0a58ab7fd9ee8d8d796d9e5a95f"}
Jul 08 03:56:25.394085 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:25.394087    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"
Jul 08 03:56:25.394663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:25.394107    2563 scope.go:117] "RemoveContainer" containerID="e7e8479b2e1e06b7a1ecec40dac0368facf5d487181689d522097ad0bd137a35"
Jul 08 03:56:25.420879 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:25.420845    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:56:25.426725 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:25.426695    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-78a872-5c756b5884-72h4t"]
Jul 08 03:56:27.386247 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:27.386216    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" path="/var/lib/kubelet/pods/f7657b28-a2f2-4a53-9a4c-8bee91d5a05b/volumes"
Jul 08 03:56:38.367162 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.367082    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" podUID="5999ba46-6f87-4379-bdd6-9661701eadaa" containerName="main" containerID="cri-o://dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776" gracePeriod=45
Jul 08 03:56:38.638186 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.638162    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:56:38.777006 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.776968    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777017    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777125    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fd9j5\" (UniqueName: \"kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777310    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm\") pod \"5999ba46-6f87-4379-bdd6-9661701eadaa\" (UID: \"5999ba46-6f87-4379-bdd6-9661701eadaa\") "
Jul 08 03:56:38.777479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777343    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache" (OuterVolumeSpecName: "model-cache") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:38.777479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777361    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home" (OuterVolumeSpecName: "home") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:38.777479 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777466    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:38.777663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777633    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-model-cache\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.777663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777648    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-home\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.777663 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.777656    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-tmp-dir\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.779716 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.779687    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:56:38.779838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.779741    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm" (OuterVolumeSpecName: "dshm") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:56:38.779838 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.779775    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5" (OuterVolumeSpecName: "kube-api-access-fd9j5") pod "5999ba46-6f87-4379-bdd6-9661701eadaa" (UID: "5999ba46-6f87-4379-bdd6-9661701eadaa"). InnerVolumeSpecName "kube-api-access-fd9j5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:56:38.878736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.878650    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5999ba46-6f87-4379-bdd6-9661701eadaa-dshm\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.878736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.878680    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fd9j5\" (UniqueName: \"kubernetes.io/projected/5999ba46-6f87-4379-bdd6-9661701eadaa-kube-api-access-fd9j5\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:38.878736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:38.878691    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5999ba46-6f87-4379-bdd6-9661701eadaa-tls-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:56:39.480750 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.480716    2563 generic.go:356] "Generic (PLEG): container finished" podID="5999ba46-6f87-4379-bdd6-9661701eadaa" containerID="dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776" exitCode=0
Jul 08 03:56:39.481248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.480759    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"
Jul 08 03:56:39.481248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.480768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" event={"ID":"5999ba46-6f87-4379-bdd6-9661701eadaa","Type":"ContainerDied","Data":"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"}
Jul 08 03:56:39.481248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.480809    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw" event={"ID":"5999ba46-6f87-4379-bdd6-9661701eadaa","Type":"ContainerDied","Data":"e1ca312e55f343c7ac220222774f929cdeb50ce33be780e63c2a94ca5a89798f"}
Jul 08 03:56:39.481248 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.480831    2563 scope.go:117] "RemoveContainer" containerID="dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"
Jul 08 03:56:39.490573 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.490554    2563 scope.go:117] "RemoveContainer" containerID="dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"
Jul 08 03:56:39.490845 ip-10-0-137-160 kubenswrapper[2563]: E0708 03:56:39.490819    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776\": container with ID starting with dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776 not found: ID does not exist" containerID="dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"
Jul 08 03:56:39.490921 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.490849    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776"} err="failed to get container status \"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776\": rpc error: code = NotFound desc = could not find container \"dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776\": container with ID starting with dba192ead24e121ca8f9ba95aa4d3208fe0a55e8ac3b0fece15acac9a6f2e776 not found: ID does not exist"
Jul 08 03:56:39.496377 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.496347    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:56:39.502015 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:39.501993    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-18addd/test-model-18addd-kserve-8d8f76c65-8vccw"]
Jul 08 03:56:41.386555 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:56:41.386519    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5999ba46-6f87-4379-bdd6-9661701eadaa" path="/var/lib/kubelet/pods/5999ba46-6f87-4379-bdd6-9661701eadaa/volumes"
Jul 08 03:57:07.293354 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.293307    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:57:07.293775 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.293548    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" podUID="71c90211-684a-4878-87f0-ecd06ee6cde5" containerName="maas-api" containerID="cri-o://047c9984914a32016aa9817e306e2365f74d710c76363aab844b200f358f675a" gracePeriod=30
Jul 08 03:57:07.649883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.649847    2563 generic.go:356] "Generic (PLEG): container finished" podID="71c90211-684a-4878-87f0-ecd06ee6cde5" containerID="047c9984914a32016aa9817e306e2365f74d710c76363aab844b200f358f675a" exitCode=0
Jul 08 03:57:07.650093 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.649922    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" event={"ID":"71c90211-684a-4878-87f0-ecd06ee6cde5","Type":"ContainerDied","Data":"047c9984914a32016aa9817e306e2365f74d710c76363aab844b200f358f675a"}
Jul 08 03:57:07.750088 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.750054    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:57:07.846284 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.846199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") pod \"71c90211-684a-4878-87f0-ecd06ee6cde5\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") "
Jul 08 03:57:07.846447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.846279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w48b5\" (UniqueName: \"kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5\") pod \"71c90211-684a-4878-87f0-ecd06ee6cde5\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") "
Jul 08 03:57:07.846447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.846352    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca\") pod \"71c90211-684a-4878-87f0-ecd06ee6cde5\" (UID: \"71c90211-684a-4878-87f0-ecd06ee6cde5\") "
Jul 08 03:57:07.846694 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.846671    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "71c90211-684a-4878-87f0-ecd06ee6cde5" (UID: "71c90211-684a-4878-87f0-ecd06ee6cde5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:57:07.848286 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.848264    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "71c90211-684a-4878-87f0-ecd06ee6cde5" (UID: "71c90211-684a-4878-87f0-ecd06ee6cde5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 03:57:07.848369 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.848350    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5" (OuterVolumeSpecName: "kube-api-access-w48b5") pod "71c90211-684a-4878-87f0-ecd06ee6cde5" (UID: "71c90211-684a-4878-87f0-ecd06ee6cde5"). InnerVolumeSpecName "kube-api-access-w48b5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:57:07.947906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.947860    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w48b5\" (UniqueName: \"kubernetes.io/projected/71c90211-684a-4878-87f0-ecd06ee6cde5-kube-api-access-w48b5\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:07.947906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.947896    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/71c90211-684a-4878-87f0-ecd06ee6cde5-openshift-service-ca\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:07.947906 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:07.947907    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/71c90211-684a-4878-87f0-ecd06ee6cde5-maas-api-tls\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:08.658114 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:08.658080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55" event={"ID":"71c90211-684a-4878-87f0-ecd06ee6cde5","Type":"ContainerDied","Data":"d9054da7596d9dba87c75be512e460a8513052b7efdcb7db645e9e43be187a27"}
Jul 08 03:57:08.658520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:08.658121    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"
Jul 08 03:57:08.658520 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:08.658162    2563 scope.go:117] "RemoveContainer" containerID="047c9984914a32016aa9817e306e2365f74d710c76363aab844b200f358f675a"
Jul 08 03:57:08.680505 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:08.680473    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:57:08.684983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:08.684946    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-18addd-856555dd5b-95g55"]
Jul 08 03:57:09.387507 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:09.387470    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71c90211-684a-4878-87f0-ecd06ee6cde5" path="/var/lib/kubelet/pods/71c90211-684a-4878-87f0-ecd06ee6cde5/volumes"
Jul 08 03:57:13.866821 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.866736    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:13.867348 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867330    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5999ba46-6f87-4379-bdd6-9661701eadaa" containerName="main"
Jul 08 03:57:13.867400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867350    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5999ba46-6f87-4379-bdd6-9661701eadaa" containerName="main"
Jul 08 03:57:13.867400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867370    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71c90211-684a-4878-87f0-ecd06ee6cde5" containerName="maas-api"
Jul 08 03:57:13.867400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867375    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="71c90211-684a-4878-87f0-ecd06ee6cde5" containerName="maas-api"
Jul 08 03:57:13.867400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867382    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" containerName="maas-api"
Jul 08 03:57:13.867400 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867388    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" containerName="maas-api"
Jul 08 03:57:13.867551 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867446    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7657b28-a2f2-4a53-9a4c-8bee91d5a05b" containerName="maas-api"
Jul 08 03:57:13.867551 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867455    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="71c90211-684a-4878-87f0-ecd06ee6cde5" containerName="maas-api"
Jul 08 03:57:13.867551 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.867462    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5999ba46-6f87-4379-bdd6-9661701eadaa" containerName="main"
Jul 08 03:57:13.871940 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.871923    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.873994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.873974    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 03:57:13.874175 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.874158    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-19508f-openshift-default-dockercfg-6lclr\""
Jul 08 03:57:13.893672 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.893637    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:13.901458 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901425    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901627 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901627 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901562    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901627 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901763 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901661    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901763 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901712    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901844 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p745q\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901886 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:13.901922 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:13.901888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.002937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.002898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.002946    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.002975    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.002994    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003023    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003181 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003129    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003569 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p745q\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003569 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003458    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003569 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003529    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003780 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003697    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.003969 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003949    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.004033 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.003961    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.005485 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.005459    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.005702 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.005685    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.010714 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.010690    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.010907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.010879    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p745q\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q\") pod \"e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") " pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.195816 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.195724    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:14.348481 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.348452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:14.349643 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:57:14.349616    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4029528_b020_42d0_913c_880f87c5f79c.slice/crio-4d9c36788acc7d1c16bf1354176050febb00329dd77a4da2dd27bdf7909e4c3d WatchSource:0}: Error finding container 4d9c36788acc7d1c16bf1354176050febb00329dd77a4da2dd27bdf7909e4c3d: Status 404 returned error can't find the container with id 4d9c36788acc7d1c16bf1354176050febb00329dd77a4da2dd27bdf7909e4c3d
Jul 08 03:57:14.351574 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.351552    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 03:57:14.696123 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:14.696091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerStarted","Data":"4d9c36788acc7d1c16bf1354176050febb00329dd77a4da2dd27bdf7909e4c3d"}
Jul 08 03:57:16.913757 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:16.913720    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:16.914038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:16.913799    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:16.914038 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:16.913830    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:17.722812 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:17.722772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerStarted","Data":"9336078e84b9595cec6643ccaa0def180603444df112b15f59f7ef49e6d97acc"}
Jul 08 03:57:17.747088 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:17.747035    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podStartSLOduration=2.185384581 podStartE2EDuration="4.747017197s" totalImagesPullingTime="2.561632616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:57:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:57:17.744668028 +0000 UTC m=+2248.853013970" watchObservedRunningTime="2026-07-08 03:57:17.747017197 +0000 UTC m=+2248.855363137"
Jul 08 03:57:18.196644 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:18.196600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:18.201736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:18.201709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:18.728307 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:18.728273    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:18.729290 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:18.729273    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:33.730390 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.730350    2563 patch_prober.go:28] interesting pod/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Jul 08 03:57:33.730811 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.730456    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Jul 08 03:57:33.844245 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.844212    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4029528-b020-42d0-913c-880f87c5f79c" containerID="9336078e84b9595cec6643ccaa0def180603444df112b15f59f7ef49e6d97acc" exitCode=0
Jul 08 03:57:33.844421 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.844273    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerDied","Data":"9336078e84b9595cec6643ccaa0def180603444df112b15f59f7ef49e6d97acc"}
Jul 08 03:57:33.844677 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.844657    2563 scope.go:117] "RemoveContainer" containerID="9336078e84b9595cec6643ccaa0def180603444df112b15f59f7ef49e6d97acc"
Jul 08 03:57:33.845749 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.845717    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:33.845807 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.845789    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:33.845843 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:33.845823    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:34.858880 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:34.858824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerStarted","Data":"02a15854994a02fe6cef138e181fbe177629541b64b1fba64a6aa85e7bbeb02a"}
Jul 08 03:57:35.196470 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:35.196360    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:35.196668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:35.196503    2563 patch_prober.go:28] interesting pod/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Jul 08 03:57:35.196668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:35.196553    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Jul 08 03:57:36.196845 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:36.196795    2563 patch_prober.go:28] interesting pod/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Jul 08 03:57:36.197521 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:36.197485    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Jul 08 03:57:37.197230 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:37.197189    2563 patch_prober.go:28] interesting pod/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Jul 08 03:57:37.197831 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:37.197796    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Jul 08 03:57:38.196178 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:38.196115    2563 patch_prober.go:28] interesting pod/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Jul 08 03:57:38.196539 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:38.196505    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Jul 08 03:57:39.201615 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:39.201584    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:39.202103 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:39.201982    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:39.202751 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:39.202734    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:46.842961 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:46.842918    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:46.843502 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:46.843203    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy" containerID="cri-o://02a15854994a02fe6cef138e181fbe177629541b64b1fba64a6aa85e7bbeb02a" gracePeriod=30
Jul 08 03:57:51.963487 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:51.963449    2563 generic.go:356] "Generic (PLEG): container finished" podID="e4029528-b020-42d0-913c-880f87c5f79c" containerID="02a15854994a02fe6cef138e181fbe177629541b64b1fba64a6aa85e7bbeb02a" exitCode=0
Jul 08 03:57:51.963921 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:51.963523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerDied","Data":"02a15854994a02fe6cef138e181fbe177629541b64b1fba64a6aa85e7bbeb02a"}
Jul 08 03:57:51.963921 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:51.963603    2563 scope.go:117] "RemoveContainer" containerID="9336078e84b9595cec6643ccaa0def180603444df112b15f59f7ef49e6d97acc"
Jul 08 03:57:52.110002 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.109933    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:52.178601 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178570    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.178601 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178605    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.178864 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178644    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.178864 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178664    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.178864 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.178864 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178857    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p745q\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.179068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178891    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.179068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.178940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.179068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179012    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket\") pod \"e4029528-b020-42d0-913c-880f87c5f79c\" (UID: \"e4029528-b020-42d0-913c-880f87c5f79c\") "
Jul 08 03:57:52.179068 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179016    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:57:52.179325 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179097    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 03:57:52.179883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179398    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4029528-b020-42d0-913c-880f87c5f79c-istiod-ca-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.179883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179518    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-socket\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.179883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179626    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:57:52.179883 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.179865    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:57:52.180183 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.180045    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:57:52.181268 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.181243    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 03:57:52.181486 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.181463    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q" (OuterVolumeSpecName: "kube-api-access-p745q") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "kube-api-access-p745q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:57:52.181888 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.181865    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 03:57:52.182120 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.182101    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4029528-b020-42d0-913c-880f87c5f79c" (UID: "e4029528-b020-42d0-913c-880f87c5f79c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 03:57:52.280102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280056    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-workload-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280087    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-credential-socket\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280096    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-istio-token\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280104    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-envoy\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280102 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280116    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4029528-b020-42d0-913c-880f87c5f79c-istio-podinfo\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280405 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280126    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p745q\" (UniqueName: \"kubernetes.io/projected/e4029528-b020-42d0-913c-880f87c5f79c-kube-api-access-p745q\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.280405 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.280134    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4029528-b020-42d0-913c-880f87c5f79c-istio-data\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 03:57:52.970036 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.970004    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"
Jul 08 03:57:52.970512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.970037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n" event={"ID":"e4029528-b020-42d0-913c-880f87c5f79c","Type":"ContainerDied","Data":"4d9c36788acc7d1c16bf1354176050febb00329dd77a4da2dd27bdf7909e4c3d"}
Jul 08 03:57:52.970512 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.970097    2563 scope.go:117] "RemoveContainer" containerID="02a15854994a02fe6cef138e181fbe177629541b64b1fba64a6aa85e7bbeb02a"
Jul 08 03:57:52.991477 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.991446    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:52.995752 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:52.995720    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-19508f-openshift-default-666f8f5fd5-gfl6n"]
Jul 08 03:57:53.389634 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:53.389606    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4029528-b020-42d0-913c-880f87c5f79c" path="/var/lib/kubelet/pods/e4029528-b020-42d0-913c-880f87c5f79c/volumes"
Jul 08 03:57:54.512971 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.512938    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 03:57:54.513571 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.513550    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:57:54.513689 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.513575    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:57:54.513746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.513697    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:57:54.513746 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.513713    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:57:54.518472 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.518448    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.520952 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.520929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-e356c0-openshift-default-dockercfg-22h2t\""
Jul 08 03:57:54.521082 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.520952    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 03:57:54.528617 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.528590    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 03:57:54.598655 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.598655 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.598960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.598960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.598960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598886    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.598960 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.599174 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.598982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.599174 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.599035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmfp5\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.599174 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.599059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699482 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699503    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699522    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699546    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699577    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmfp5\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699729    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699772    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.699937 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.699811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.700094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.700032    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.700094 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.700063    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.700363 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.700344    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.700408 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.700379    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.700447 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.700434    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.702092 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.702071    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.702681 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.702662    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.706668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.706641    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.707099 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.707078    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmfp5\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5\") pod \"e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") " pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.830963 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.830930    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:54.974926 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.974883    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 03:57:54.976060 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:57:54.976035    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5186754_6d56_477a_b986_12ca41894817.slice/crio-7e5c8a25cd8205458e43ea9c95f7d73c0b9c9d2f1898c3855c74a59aff412ed7 WatchSource:0}: Error finding container 7e5c8a25cd8205458e43ea9c95f7d73c0b9c9d2f1898c3855c74a59aff412ed7: Status 404 returned error can't find the container with id 7e5c8a25cd8205458e43ea9c95f7d73c0b9c9d2f1898c3855c74a59aff412ed7
Jul 08 03:57:54.978413 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.978363    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:54.978518 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.978452    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:54.978518 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.978498    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:57:54.987440 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:54.987407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerStarted","Data":"7e5c8a25cd8205458e43ea9c95f7d73c0b9c9d2f1898c3855c74a59aff412ed7"}
Jul 08 03:57:55.995656 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:55.995616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerStarted","Data":"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"}
Jul 08 03:57:56.017009 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:56.016954    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podStartSLOduration=2.016937836 podStartE2EDuration="2.016937836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:57:56.015376898 +0000 UTC m=+2287.123722840" watchObservedRunningTime="2026-07-08 03:57:56.016937836 +0000 UTC m=+2287.125283776"
Jul 08 03:57:56.831535 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:56.831494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:56.836453 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:56.836426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:57.002106 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:57.002071    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:57:57.002998 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:57:57.002981    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:58:12.004537 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.004493    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 08 03:58:12.004983 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.004563    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 08 03:58:12.099004 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.098966    2563 generic.go:356] "Generic (PLEG): container finished" podID="f5186754-6d56-477a-b986-12ca41894817" containerID="6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539" exitCode=0
Jul 08 03:58:12.099185 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.099008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerDied","Data":"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"}
Jul 08 03:58:12.099474 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.099460    2563 scope.go:117] "RemoveContainer" containerID="6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"
Jul 08 03:58:12.100621 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.100588    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:58:12.100707 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.100688    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:58:12.100755 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:12.100742    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 08 03:58:13.111655 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:13.111612    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerStarted","Data":"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"}
Jul 08 03:58:13.831449 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:13.831417    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:58:13.831668 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:13.831646    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 08 03:58:13.831725 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:13.831699    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 08 03:58:14.831511 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:14.831471    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 08 03:58:14.831933 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:14.831527    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 08 03:58:15.832112 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:15.832068    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 08 03:58:15.832567 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:15.832133    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 08 03:58:16.832330 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:16.832294    2563 patch_prober.go:28] interesting pod/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body=
Jul 08 03:58:16.832743 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:16.832354    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused"
Jul 08 03:58:17.836127 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:17.836098    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:58:17.836590 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:17.836576    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:58:17.837246 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:17.837228    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 03:58:38.349056 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.349025    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 03:58:38.349487 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.349457    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:58:38.349487 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.349472    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4029528-b020-42d0-913c-880f87c5f79c" containerName="istio-proxy"
Jul 08 03:58:38.352907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.352885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.355231 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.355201    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"kube-root-ca.crt\""
Jul 08 03:58:38.355789 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.355769    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"default-dockercfg-8kqg9\""
Jul 08 03:58:38.355907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.355808    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"openshift-service-ca.crt\""
Jul 08 03:58:38.355907 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.355831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b98412\"/\"auth-test-model-b98412-kserve-self-signed-certs\""
Jul 08 03:58:38.363799 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.363778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 03:58:38.444401 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444362    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.444597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444416    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.444597 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444557    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.444693 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqn4x\" (UniqueName: \"kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.444693 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444631    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.444693 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.444660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545391 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545350    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545622 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545622 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqn4x\" (UniqueName: \"kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545622 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545622 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545588    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545842 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545646    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545842 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545769    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545842 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545817    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.545999 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.545911    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.548050 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.548029    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.548164 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.548099    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.554263 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.554237    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqn4x\" (UniqueName: \"kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x\") pod \"auth-test-model-b98412-kserve-7c75c46dc-s2jzp\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") " pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.662721 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.662636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:38.811711 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:38.811666    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 03:58:38.813561 ip-10-0-137-160 kubenswrapper[2563]: W0708 03:58:38.813531    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4296e6e6_10c5_41e8_8957_f903dbc2f9db.slice/crio-fce29b2538c1012f281f6a8fb7613422c8055c1d897df7151834e1940ba65b84 WatchSource:0}: Error finding container fce29b2538c1012f281f6a8fb7613422c8055c1d897df7151834e1940ba65b84: Status 404 returned error can't find the container with id fce29b2538c1012f281f6a8fb7613422c8055c1d897df7151834e1940ba65b84
Jul 08 03:58:39.276489 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:39.276448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" event={"ID":"4296e6e6-10c5-41e8-8957-f903dbc2f9db","Type":"ContainerStarted","Data":"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"}
Jul 08 03:58:39.276680 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:39.276502    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" event={"ID":"4296e6e6-10c5-41e8-8957-f903dbc2f9db","Type":"ContainerStarted","Data":"fce29b2538c1012f281f6a8fb7613422c8055c1d897df7151834e1940ba65b84"}
Jul 08 03:58:39.293736 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:39.293672    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" podStartSLOduration=1.293650124 podStartE2EDuration="1.293650124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 03:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 03:58:39.292589189 +0000 UTC m=+2330.400935128" watchObservedRunningTime="2026-07-08 03:58:39.293650124 +0000 UTC m=+2330.401996065"
Jul 08 03:58:48.662994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:48.662916    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:48.662994 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:48.662972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:48.675645 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:48.675620    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:58:49.350839 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:58:49.350807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 03:59:50.177426 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:59:50.177397    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:59:50.182176 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:59:50.182134    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 03:59:50.184111 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:59:50.184094    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 03:59:50.188524 ip-10-0-137-160 kubenswrapper[2563]: I0708 03:59:50.188499    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 04:00:00.133422 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.133386    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"]
Jul 08 04:00:00.137024 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.137001    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"]
Jul 08 04:00:00.137212 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.137156    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:00.140565 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.140539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:00.148079 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.148002    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"]
Jul 08 04:00:00.151122 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.151095    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"]
Jul 08 04:00:00.252878 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.252845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5hwj\" (UniqueName: \"kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj\") pod \"maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv\" (UID: \"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:00.253061 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.252908    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rn5dp\" (UniqueName: \"kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp\") pod \"maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx\" (UID: \"179923b5-0534-445f-88ef-69350843b6f8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:00.353896 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.353857    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rn5dp\" (UniqueName: \"kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp\") pod \"maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx\" (UID: \"179923b5-0534-445f-88ef-69350843b6f8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:00.354077 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.353952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5hwj\" (UniqueName: \"kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj\") pod \"maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv\" (UID: \"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:00.363009 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.362972    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rn5dp\" (UniqueName: \"kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp\") pod \"maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx\" (UID: \"179923b5-0534-445f-88ef-69350843b6f8\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:00.363009 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.363002    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5hwj\" (UniqueName: \"kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj\") pod \"maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv\" (UID: \"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:00.452760 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.452679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:00.460809 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.460789    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:00.612514 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.612482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"]
Jul 08 04:00:00.613100 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:00:00.613071    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f1cadbe_f658_4ef2_afd9_d18a9e266b3b.slice/crio-16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1 WatchSource:0}: Error finding container 16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1: Status 404 returned error can't find the container with id 16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1
Jul 08 04:00:00.632550 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.632521    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"]
Jul 08 04:00:00.633727 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:00:00.633700    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179923b5_0534_445f_88ef_69350843b6f8.slice/crio-e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306 WatchSource:0}: Error finding container e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306: Status 404 returned error can't find the container with id e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306
Jul 08 04:00:00.766233 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.766119    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv" event={"ID":"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b","Type":"ContainerStarted","Data":"16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1"}
Jul 08 04:00:00.767360 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:00.767336    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx" event={"ID":"179923b5-0534-445f-88ef-69350843b6f8","Type":"ContainerStarted","Data":"e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306"}
Jul 08 04:00:01.775740 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:01.775703    2563 generic.go:356] "Generic (PLEG): container finished" podID="7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" containerID="17d6c561aab6982adfca5317d83c6af47fcdc248669ef6df0c203b7b5248e072" exitCode=0
Jul 08 04:00:01.776175 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:01.775784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv" event={"ID":"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b","Type":"ContainerDied","Data":"17d6c561aab6982adfca5317d83c6af47fcdc248669ef6df0c203b7b5248e072"}
Jul 08 04:00:01.777316 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:01.777296    2563 generic.go:356] "Generic (PLEG): container finished" podID="179923b5-0534-445f-88ef-69350843b6f8" containerID="90eb091afcae08da3f638e02515c337a58b119b6bf5779cf0ab664ece3875d2e" exitCode=0
Jul 08 04:00:01.777429 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:01.777378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx" event={"ID":"179923b5-0534-445f-88ef-69350843b6f8","Type":"ContainerDied","Data":"90eb091afcae08da3f638e02515c337a58b119b6bf5779cf0ab664ece3875d2e"}
Jul 08 04:00:02.924797 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:02.924768    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:02.962478 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:02.962448    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:02.981847 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:02.981767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rn5dp\" (UniqueName: \"kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp\") pod \"179923b5-0534-445f-88ef-69350843b6f8\" (UID: \"179923b5-0534-445f-88ef-69350843b6f8\") "
Jul 08 04:00:02.985278 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:02.985235    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp" (OuterVolumeSpecName: "kube-api-access-rn5dp") pod "179923b5-0534-445f-88ef-69350843b6f8" (UID: "179923b5-0534-445f-88ef-69350843b6f8"). InnerVolumeSpecName "kube-api-access-rn5dp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:00:03.082724 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.082694    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5hwj\" (UniqueName: \"kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj\") pod \"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b\" (UID: \"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b\") "
Jul 08 04:00:03.083031 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.083011    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rn5dp\" (UniqueName: \"kubernetes.io/projected/179923b5-0534-445f-88ef-69350843b6f8-kube-api-access-rn5dp\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:03.085014 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.084978    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj" (OuterVolumeSpecName: "kube-api-access-h5hwj") pod "7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" (UID: "7f1cadbe-f658-4ef2-afd9-d18a9e266b3b"). InnerVolumeSpecName "kube-api-access-h5hwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:00:03.184356 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.184317    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5hwj\" (UniqueName: \"kubernetes.io/projected/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b-kube-api-access-h5hwj\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:03.791903 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.791817    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"
Jul 08 04:00:03.792062 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.791821    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv" event={"ID":"7f1cadbe-f658-4ef2-afd9-d18a9e266b3b","Type":"ContainerDied","Data":"16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1"}
Jul 08 04:00:03.792062 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.791949    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16f3fa5ed220e0a48951ea43e781557ea7bc5b7bfaec8e86ad90e40efe4252a1"
Jul 08 04:00:03.793344 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.793325    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"
Jul 08 04:00:03.793459 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.793371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx" event={"ID":"179923b5-0534-445f-88ef-69350843b6f8","Type":"ContainerDied","Data":"e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306"}
Jul 08 04:00:03.793459 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:03.793409    2563 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4d90f4be1709fea48d3c46f1e6778474e0c08796bda77e179b20398cdffa306"
Jul 08 04:00:35.051058 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:35.051022    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 04:00:48.846684 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.846646    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:00:48.847088 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847074    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="179923b5-0534-445f-88ef-69350843b6f8" containerName="cleanup"
Jul 08 04:00:48.847133 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847090    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="179923b5-0534-445f-88ef-69350843b6f8" containerName="cleanup"
Jul 08 04:00:48.847133 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847107    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" containerName="cleanup"
Jul 08 04:00:48.847133 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847112    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" containerName="cleanup"
Jul 08 04:00:48.847272 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847193    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" containerName="cleanup"
Jul 08 04:00:48.847272 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.847203    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="179923b5-0534-445f-88ef-69350843b6f8" containerName="cleanup"
Jul 08 04:00:48.850592 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.850573    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.852601 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.852579    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"kube-root-ca.crt\""
Jul 08 04:00:48.853295 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.853277    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"openshift-service-ca.crt\""
Jul 08 04:00:48.853410 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.853314    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"default-dockercfg-zd2gp\""
Jul 08 04:00:48.853410 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.853345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"sub-test-model-e356c0-kserve-self-signed-certs\""
Jul 08 04:00:48.857620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.857596    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:00:48.927270 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.927270 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927273    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.927527 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927305    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.927527 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.927645 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927550    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:48.927645 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:48.927588    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vggq5\" (UniqueName: \"kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028384 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028597 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vggq5\" (UniqueName: \"kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028597 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028440    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028597 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028799 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028799 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028743    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028799 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028782    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.028951 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.028896    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.029047 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.029029    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.030963 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.030936    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.031158 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.031126    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.036637 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.036618    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vggq5\" (UniqueName: \"kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5\") pod \"sub-test-model-e356c0-kserve-588ff8fcf5-ks82z\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") " pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.161632 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.161540    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:49.316362 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:49.316335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:00:49.318376 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:00:49.318346    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod93b3faba_86d1_4979_811f_8d2965b2323a.slice/crio-1d153b4086493fdce7b7c2dbee9c846a118036ab80b115d752dc57bf74a09938 WatchSource:0}: Error finding container 1d153b4086493fdce7b7c2dbee9c846a118036ab80b115d752dc57bf74a09938: Status 404 returned error can't find the container with id 1d153b4086493fdce7b7c2dbee9c846a118036ab80b115d752dc57bf74a09938
Jul 08 04:00:50.073280 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.073236    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" podUID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" containerName="main" containerID="cri-o://eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77" gracePeriod=45
Jul 08 04:00:50.083366 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.083327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" event={"ID":"93b3faba-86d1-4979-811f-8d2965b2323a","Type":"ContainerStarted","Data":"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"}
Jul 08 04:00:50.083524 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.083384    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" event={"ID":"93b3faba-86d1-4979-811f-8d2965b2323a","Type":"ContainerStarted","Data":"1d153b4086493fdce7b7c2dbee9c846a118036ab80b115d752dc57bf74a09938"}
Jul 08 04:00:50.102770 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.102717    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" podStartSLOduration=2.102701792 podStartE2EDuration="2.102701792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:00:50.10044587 +0000 UTC m=+2461.208791810" watchObservedRunningTime="2026-07-08 04:00:50.102701792 +0000 UTC m=+2461.211047731"
Jul 08 04:00:50.341224 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.341199    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 04:00:50.445257 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445221    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445430 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445430 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445322    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445430 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445430 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqn4x\" (UniqueName: \"kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445467    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs\") pod \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\" (UID: \"4296e6e6-10c5-41e8-8957-f903dbc2f9db\") "
Jul 08 04:00:50.445620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445540    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:50.445620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445569    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home" (OuterVolumeSpecName: "home") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:50.445781 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445661    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache" (OuterVolumeSpecName: "model-cache") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:50.446000 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.445974    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tmp-dir\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:50.446302 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.446004    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-home\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:50.446302 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.446019    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-model-cache\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:50.447830 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.447799    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:00:50.447924 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.447829    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm" (OuterVolumeSpecName: "dshm") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:00:50.447924 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.447903    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x" (OuterVolumeSpecName: "kube-api-access-gqn4x") pod "4296e6e6-10c5-41e8-8957-f903dbc2f9db" (UID: "4296e6e6-10c5-41e8-8957-f903dbc2f9db"). InnerVolumeSpecName "kube-api-access-gqn4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:00:50.546843 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.546807    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4296e6e6-10c5-41e8-8957-f903dbc2f9db-tls-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:50.546843 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.546834    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4296e6e6-10c5-41e8-8957-f903dbc2f9db-dshm\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:50.546843 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:50.546844    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqn4x\" (UniqueName: \"kubernetes.io/projected/4296e6e6-10c5-41e8-8957-f903dbc2f9db-kube-api-access-gqn4x\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:00:51.091100 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.091060    2563 generic.go:356] "Generic (PLEG): container finished" podID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" containerID="eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77" exitCode=0
Jul 08 04:00:51.091556 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.091110    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"
Jul 08 04:00:51.091556 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.091148    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" event={"ID":"4296e6e6-10c5-41e8-8957-f903dbc2f9db","Type":"ContainerDied","Data":"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"}
Jul 08 04:00:51.091556 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.091200    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp" event={"ID":"4296e6e6-10c5-41e8-8957-f903dbc2f9db","Type":"ContainerDied","Data":"fce29b2538c1012f281f6a8fb7613422c8055c1d897df7151834e1940ba65b84"}
Jul 08 04:00:51.091556 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.091223    2563 scope.go:117] "RemoveContainer" containerID="eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"
Jul 08 04:00:51.101817 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.101801    2563 scope.go:117] "RemoveContainer" containerID="eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"
Jul 08 04:00:51.102113 ip-10-0-137-160 kubenswrapper[2563]: E0708 04:00:51.102091    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77\": container with ID starting with eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77 not found: ID does not exist" containerID="eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"
Jul 08 04:00:51.102193 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.102126    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77"} err="failed to get container status \"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77\": rpc error: code = NotFound desc = could not find container \"eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77\": container with ID starting with eeaae3456c443a3fd2c61ce849e498c436fff06790f062a23fd3221421f89e77 not found: ID does not exist"
Jul 08 04:00:51.112660 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.112634    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 04:00:51.119048 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.119025    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b98412/auth-test-model-b98412-kserve-7c75c46dc-s2jzp"]
Jul 08 04:00:51.387281 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:51.387198    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" path="/var/lib/kubelet/pods/4296e6e6-10c5-41e8-8957-f903dbc2f9db/volumes"
Jul 08 04:00:59.162229 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:59.162187    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:59.162229 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:59.162238    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:00:59.174993 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:00:59.174964    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:01:00.159395 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:01:00.159361    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:01:53.830708 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:01:53.830674    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:02:08.445473 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.445421    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:02:08.446165 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.446113    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" containerName="main"
Jul 08 04:02:08.446165 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.446167    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" containerName="main"
Jul 08 04:02:08.446342 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.446322    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4296e6e6-10c5-41e8-8957-f903dbc2f9db" containerName="main"
Jul 08 04:02:08.449410 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.449391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.451837 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.451818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-e356c0\"/\"rate-test-model-e356c0-kserve-self-signed-certs\""
Jul 08 04:02:08.458291 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.458266    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:02:08.478718 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.478684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.478906 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.478754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.478906 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.478813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.479017 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.478932    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.479017 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.478987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.479110 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.479074    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2fvb\" (UniqueName: \"kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580480 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580683 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580518    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2fvb\" (UniqueName: \"kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580683 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580547    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580683 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580683 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.580683 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.580683    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.581122 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.581057    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.581285 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.581199    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.581461 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.581278    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.583167 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.583114    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.583392 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.583373    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.590596 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.590572    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2fvb\" (UniqueName: \"kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb\") pod \"rate-test-model-e356c0-kserve-6b49db5ddc-hzck4\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") " pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.758889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.758787    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:08.852258 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.852071    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" podUID="93b3faba-86d1-4979-811f-8d2965b2323a" containerName="main" containerID="cri-o://86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3" gracePeriod=45
Jul 08 04:02:08.907724 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:08.907698    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:02:08.914365 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:02:08.914329    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36ea6388_0dbc_497b_aef6_8baff82d8819.slice/crio-d8baf74ef753c6b23af509979f833b2bee91ca334ec775d5ff75d486658fa4eb WatchSource:0}: Error finding container d8baf74ef753c6b23af509979f833b2bee91ca334ec775d5ff75d486658fa4eb: Status 404 returned error can't find the container with id d8baf74ef753c6b23af509979f833b2bee91ca334ec775d5ff75d486658fa4eb
Jul 08 04:02:09.103279 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.103251    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:02:09.187522 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187484    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187717 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187551    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vggq5\" (UniqueName: \"kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187717 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187717 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187664    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187717 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187688    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187936 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187757    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home\") pod \"93b3faba-86d1-4979-811f-8d2965b2323a\" (UID: \"93b3faba-86d1-4979-811f-8d2965b2323a\") "
Jul 08 04:02:09.187936 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.187901    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:09.188106 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.188080    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache" (OuterVolumeSpecName: "model-cache") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:09.188106 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.188091    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home" (OuterVolumeSpecName: "home") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:09.188440 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.188164    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-tmp-dir\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.189952 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.189926    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:02:09.190059 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.189962    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5" (OuterVolumeSpecName: "kube-api-access-vggq5") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "kube-api-access-vggq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:02:09.190115 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.190099    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm" (OuterVolumeSpecName: "dshm") pod "93b3faba-86d1-4979-811f-8d2965b2323a" (UID: "93b3faba-86d1-4979-811f-8d2965b2323a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:02:09.289497 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.289421    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/93b3faba-86d1-4979-811f-8d2965b2323a-tls-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.289497 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.289450    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-model-cache\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.289497 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.289458    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-home\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.289497 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.289465    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/93b3faba-86d1-4979-811f-8d2965b2323a-dshm\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.289497 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.289474    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vggq5\" (UniqueName: \"kubernetes.io/projected/93b3faba-86d1-4979-811f-8d2965b2323a-kube-api-access-vggq5\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:02:09.572243 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.572210    2563 generic.go:356] "Generic (PLEG): container finished" podID="93b3faba-86d1-4979-811f-8d2965b2323a" containerID="86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3" exitCode=0
Jul 08 04:02:09.572711 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.572260    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"
Jul 08 04:02:09.572711 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.572297    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" event={"ID":"93b3faba-86d1-4979-811f-8d2965b2323a","Type":"ContainerDied","Data":"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"}
Jul 08 04:02:09.572711 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.572344    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z" event={"ID":"93b3faba-86d1-4979-811f-8d2965b2323a","Type":"ContainerDied","Data":"1d153b4086493fdce7b7c2dbee9c846a118036ab80b115d752dc57bf74a09938"}
Jul 08 04:02:09.572711 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.572367    2563 scope.go:117] "RemoveContainer" containerID="86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"
Jul 08 04:02:09.574073 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.573934    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" event={"ID":"36ea6388-0dbc-497b-aef6-8baff82d8819","Type":"ContainerStarted","Data":"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"}
Jul 08 04:02:09.574207 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.574092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" event={"ID":"36ea6388-0dbc-497b-aef6-8baff82d8819","Type":"ContainerStarted","Data":"d8baf74ef753c6b23af509979f833b2bee91ca334ec775d5ff75d486658fa4eb"}
Jul 08 04:02:09.583191 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.583164    2563 scope.go:117] "RemoveContainer" containerID="86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"
Jul 08 04:02:09.583483 ip-10-0-137-160 kubenswrapper[2563]: E0708 04:02:09.583465    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3\": container with ID starting with 86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3 not found: ID does not exist" containerID="86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"
Jul 08 04:02:09.583582 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.583492    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3"} err="failed to get container status \"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3\": rpc error: code = NotFound desc = could not find container \"86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3\": container with ID starting with 86d7260bd9c3d0a66d6ad3f26a32d10dfcce0bc3f6a2a017fea005d16c0bb4c3 not found: ID does not exist"
Jul 08 04:02:09.596636 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.596611    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:02:09.601756 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.601732    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/sub-test-model-e356c0-kserve-588ff8fcf5-ks82z"]
Jul 08 04:02:09.616482 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:09.616423    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" podStartSLOduration=1.616403644 podStartE2EDuration="1.616403644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:02:09.614327784 +0000 UTC m=+2540.722673716" watchObservedRunningTime="2026-07-08 04:02:09.616403644 +0000 UTC m=+2540.724749586"
Jul 08 04:02:11.386667 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:11.386638    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93b3faba-86d1-4979-811f-8d2965b2323a" path="/var/lib/kubelet/pods/93b3faba-86d1-4979-811f-8d2965b2323a/volumes"
Jul 08 04:02:18.758937 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:18.758898    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:18.759370 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:18.759003    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:18.771573 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:18.771551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:02:19.649106 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:02:19.649079    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:03:21.823876 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:21.823842    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:03:24.382679 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:24.382644    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"]
Jul 08 04:03:24.385509 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:24.385483    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-e356c0-29724720-ddlkv"]
Jul 08 04:03:25.387361 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:25.387324    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f1cadbe-f658-4ef2-afd9-d18a9e266b3b" path="/var/lib/kubelet/pods/7f1cadbe-f658-4ef2-afd9-d18a9e266b3b/volumes"
Jul 08 04:03:36.844606 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:36.844549    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" podUID="36ea6388-0dbc-497b-aef6-8baff82d8819" containerName="main" containerID="cri-o://5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498" gracePeriod=45
Jul 08 04:03:37.107566 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.107496    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:03:37.120828 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.120796    2563 generic.go:356] "Generic (PLEG): container finished" podID="36ea6388-0dbc-497b-aef6-8baff82d8819" containerID="5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498" exitCode=0
Jul 08 04:03:37.120975 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.120834    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"
Jul 08 04:03:37.120975 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.120875    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" event={"ID":"36ea6388-0dbc-497b-aef6-8baff82d8819","Type":"ContainerDied","Data":"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"}
Jul 08 04:03:37.120975 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.120920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4" event={"ID":"36ea6388-0dbc-497b-aef6-8baff82d8819","Type":"ContainerDied","Data":"d8baf74ef753c6b23af509979f833b2bee91ca334ec775d5ff75d486658fa4eb"}
Jul 08 04:03:37.120975 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.120937    2563 scope.go:117] "RemoveContainer" containerID="5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"
Jul 08 04:03:37.131712 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.131687    2563 scope.go:117] "RemoveContainer" containerID="5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"
Jul 08 04:03:37.132042 ip-10-0-137-160 kubenswrapper[2563]: E0708 04:03:37.132015    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498\": container with ID starting with 5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498 not found: ID does not exist" containerID="5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"
Jul 08 04:03:37.132158 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.132053    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498"} err="failed to get container status \"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498\": rpc error: code = NotFound desc = could not find container \"5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498\": container with ID starting with 5192f37131977f656af29f68e49840269e6fd997d63ccdeb87edec725df93498 not found: ID does not exist"
Jul 08 04:03:37.268314 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268279    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268464 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268378    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268464 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268405    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268464 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268421    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268464 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268442    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268679 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268520    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2fvb\" (UniqueName: \"kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb\") pod \"36ea6388-0dbc-497b-aef6-8baff82d8819\" (UID: \"36ea6388-0dbc-497b-aef6-8baff82d8819\") "
Jul 08 04:03:37.268679 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268550    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache" (OuterVolumeSpecName: "model-cache") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:37.268790 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268710    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home" (OuterVolumeSpecName: "home") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:37.268790 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268720    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:37.268888 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268864    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-model-cache\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.268888 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268877    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-tmp-dir\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.268888 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.268886    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-home\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.270852 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.270820    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb" (OuterVolumeSpecName: "kube-api-access-f2fvb") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "kube-api-access-f2fvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:03:37.270852 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.270837    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm" (OuterVolumeSpecName: "dshm") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:03:37.270999 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.270894    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36ea6388-0dbc-497b-aef6-8baff82d8819" (UID: "36ea6388-0dbc-497b-aef6-8baff82d8819"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 04:03:37.369836 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.369750    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2fvb\" (UniqueName: \"kubernetes.io/projected/36ea6388-0dbc-497b-aef6-8baff82d8819-kube-api-access-f2fvb\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.369836 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.369777    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36ea6388-0dbc-497b-aef6-8baff82d8819-tls-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.369836 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.369786    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36ea6388-0dbc-497b-aef6-8baff82d8819-dshm\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:03:37.440481 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.440448    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:03:37.445441 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:37.445412    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-e356c0/rate-test-model-e356c0-kserve-6b49db5ddc-hzck4"]
Jul 08 04:03:39.388494 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:03:39.388457    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36ea6388-0dbc-497b-aef6-8baff82d8819" path="/var/lib/kubelet/pods/36ea6388-0dbc-497b-aef6-8baff82d8819/volumes"
Jul 08 04:04:06.522608 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:06.522577    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 04:04:06.523196 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:06.522807    2563 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy" containerID="cri-o://3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d" gracePeriod=30
Jul 08 04:04:07.388287 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:07.388251    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"]
Jul 08 04:04:07.388287 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:07.388291    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-b98412-29724720-ddrtx"]
Jul 08 04:04:09.388155 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:09.388099    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="179923b5-0534-445f-88ef-69350843b6f8" path="/var/lib/kubelet/pods/179923b5-0534-445f-88ef-69350843b6f8/volumes"
Jul 08 04:04:11.781452 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.781424    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 04:04:11.886804 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886769    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.886804 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886806    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886829    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886911    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886951    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.886981    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887045    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887084 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887084    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmfp5\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887418 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887134    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket\") pod \"f5186754-6d56-477a-b986-12ca41894817\" (UID: \"f5186754-6d56-477a-b986-12ca41894817\") "
Jul 08 04:04:11.887418 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887236    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:04:11.887418 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887280    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:04:11.887575 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887541    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-credential-socket\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.887575 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887562    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-certs\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.887777 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887752    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 04:04:11.887902 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887880    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:04:11.888041 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.887920    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:04:11.889594 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.889562    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 04:04:11.889784 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.889760    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 04:04:11.890095 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.890070    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:04:11.890095 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.890082    2563 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5" (OuterVolumeSpecName: "kube-api-access-hmfp5") pod "f5186754-6d56-477a-b986-12ca41894817" (UID: "f5186754-6d56-477a-b986-12ca41894817"). InnerVolumeSpecName "kube-api-access-hmfp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 04:04:11.988552 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988512    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-workload-socket\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988552 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988545    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5186754-6d56-477a-b986-12ca41894817-istio-podinfo\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988552 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988554    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-data\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988552 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988562    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-istio-token\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988811 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988571    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5186754-6d56-477a-b986-12ca41894817-istio-envoy\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988811 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988578    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5186754-6d56-477a-b986-12ca41894817-istiod-ca-cert\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:11.988811 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:11.988586    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmfp5\" (UniqueName: \"kubernetes.io/projected/f5186754-6d56-477a-b986-12ca41894817-kube-api-access-hmfp5\") on node \"ip-10-0-137-160.ec2.internal\" DevicePath \"\""
Jul 08 04:04:12.341710 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.341679    2563 generic.go:356] "Generic (PLEG): container finished" podID="f5186754-6d56-477a-b986-12ca41894817" containerID="3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d" exitCode=0
Jul 08 04:04:12.341889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.341728    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"
Jul 08 04:04:12.341889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.341730    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerDied","Data":"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"}
Jul 08 04:04:12.341889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.341835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv" event={"ID":"f5186754-6d56-477a-b986-12ca41894817","Type":"ContainerDied","Data":"7e5c8a25cd8205458e43ea9c95f7d73c0b9c9d2f1898c3855c74a59aff412ed7"}
Jul 08 04:04:12.341889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.341856    2563 scope.go:117] "RemoveContainer" containerID="3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"
Jul 08 04:04:12.356708 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.356114    2563 scope.go:117] "RemoveContainer" containerID="6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"
Jul 08 04:04:12.366253 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.366228    2563 scope.go:117] "RemoveContainer" containerID="3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"
Jul 08 04:04:12.366559 ip-10-0-137-160 kubenswrapper[2563]: E0708 04:04:12.366538    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d\": container with ID starting with 3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d not found: ID does not exist" containerID="3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"
Jul 08 04:04:12.366622 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.366567    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d"} err="failed to get container status \"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d\": rpc error: code = NotFound desc = could not find container \"3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d\": container with ID starting with 3e475d976f4d1b63991b1886571b20e2e9fd2fc6bc67c2159cc54489d54f2e4d not found: ID does not exist"
Jul 08 04:04:12.366622 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.366586    2563 scope.go:117] "RemoveContainer" containerID="6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"
Jul 08 04:04:12.366883 ip-10-0-137-160 kubenswrapper[2563]: E0708 04:04:12.366852    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539\": container with ID starting with 6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539 not found: ID does not exist" containerID="6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"
Jul 08 04:04:12.366977 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.366888    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539"} err="failed to get container status \"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539\": rpc error: code = NotFound desc = could not find container \"6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539\": container with ID starting with 6315c07096afa4863ab5b9d013a1abb3c7ee7b4c20ad7b89070851c145c68539 not found: ID does not exist"
Jul 08 04:04:12.368818 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.368789    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 04:04:12.371900 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:12.371878    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-e356c0-openshift-default-59dd67f65f-q2vkv"]
Jul 08 04:04:13.388836 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:13.388797    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5186754-6d56-477a-b986-12ca41894817" path="/var/lib/kubelet/pods/f5186754-6d56-477a-b986-12ca41894817/volumes"
Jul 08 04:04:14.228357 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:14.228324    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b46ff5d4-rk57q_f077e16d-a017-48e7-a15c-f8c1420ef50b/authorino/0.log"
Jul 08 04:04:18.691971 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:18.691929    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zdd8t_45057a34-5fa4-44bf-bfa3-68afa7748696/manager/0.log"
Jul 08 04:04:18.812937 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:18.812904    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rr99j_c4f1f2e3-aa65-48e1-ad25-c2f301658ed1/manager/0.log"
Jul 08 04:04:18.930881 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:18.930835    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bcbd5b4c9-2dd2c_d752bb66-9c4f-4ea7-ae8f-31af63170554/maas-api/0.log"
Jul 08 04:04:19.040986 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.040901    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724705-sjrf8_58723121-7b6d-4fbb-9ac8-eee93d38dfb9/cleanup/0.log"
Jul 08 04:04:19.268460 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.268409    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b857d6d59-k95zk_632d8a6d-259b-4780-9571-3aa34bcfe8f5/manager/0.log"
Jul 08 04:04:19.377892 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.377861    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vwb5b_583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa/server/0.log"
Jul 08 04:04:19.498455 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.498421    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bl9cn_115aced2-45b7-4e4b-9ddb-4424117fac5b/manager/0.log"
Jul 08 04:04:19.870390 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.870359    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c_c87e613a-f280-4a78-8f4b-fd8579bd264e/copy-manifests/0.log"
Jul 08 04:04:19.877021 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.876998    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c_c87e613a-f280-4a78-8f4b-fd8579bd264e/manager/0.log"
Jul 08 04:04:19.991008 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:19.990975    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-2j4bv_53cbffe1-3a17-4d0b-b882-87ef081af3fd/postgres/0.log"
Jul 08 04:04:20.755559 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.755531    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/util/0.log"
Jul 08 04:04:20.766944 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.766926    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/pull/0.log"
Jul 08 04:04:20.776669 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.776649    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/extract/0.log"
Jul 08 04:04:20.887992 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.887958    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/extract/0.log"
Jul 08 04:04:20.895909 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.895888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/util/0.log"
Jul 08 04:04:20.902745 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:20.902725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/pull/0.log"
Jul 08 04:04:21.032723 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.032643    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/util/0.log"
Jul 08 04:04:21.049970 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.049938    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/pull/0.log"
Jul 08 04:04:21.056277 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.056255    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/extract/0.log"
Jul 08 04:04:21.163052 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.163017    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/pull/0.log"
Jul 08 04:04:21.170430 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.170407    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/extract/0.log"
Jul 08 04:04:21.176471 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.176447    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/util/0.log"
Jul 08 04:04:21.302742 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.302711    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b46ff5d4-rk57q_f077e16d-a017-48e7-a15c-f8c1420ef50b/authorino/0.log"
Jul 08 04:04:21.416598 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.416561    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z9p6p_d62ec88a-5e14-4243-836b-8b0e6f70daff/manager/0.log"
Jul 08 04:04:21.523371 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.523341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-95jls_fdf922e1-1caa-41c6-9f2a-2817899411a0/manager/0.log"
Jul 08 04:04:21.749864 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.749754    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mnhl_8e852069-5b7e-46aa-9171-d3cd658d5a40/registry-server/0.log"
Jul 08 04:04:21.870563 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:21.870530    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9z84k_8d650c5b-7758-45a7-bbe6-6b2ebeabd132/manager/0.log"
Jul 08 04:04:22.100931 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:22.100893    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ztm2h_cb870b59-f919-45ce-9977-38d8ca77dfaf/manager/0.log"
Jul 08 04:04:22.666441 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:22.666416    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-b7drh_2612f803-f607-4d02-bc14-0ba3328c8d03/kube-auth-proxy/0.log"
Jul 08 04:04:23.132227 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:23.132195    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6rfwn_e480dfee-6196-42e2-abc0-9e0b027f4db5/payload-processing/0.log"
Jul 08 04:04:23.243805 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:23.243775    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cf9c66499-8vtcz_aff7f08a-06ac-41bb-9e76-14f8670b4c84/router/0.log"
Jul 08 04:04:23.582162 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:23.582115    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-pcckk_efcec4d9-7c63-4550-89f1-58363b298c46/main/0.log"
Jul 08 04:04:27.659547 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.659501    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hvjg8/must-gather-4kxtx"]
Jul 08 04:04:27.660106 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660085    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660111    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660125    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93b3faba-86d1-4979-811f-8d2965b2323a" containerName="main"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660132    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="93b3faba-86d1-4979-811f-8d2965b2323a" containerName="main"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660161    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660170    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660183    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36ea6388-0dbc-497b-aef6-8baff82d8819" containerName="main"
Jul 08 04:04:27.660245 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660191    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="36ea6388-0dbc-497b-aef6-8baff82d8819" containerName="main"
Jul 08 04:04:27.660599 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660332    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="93b3faba-86d1-4979-811f-8d2965b2323a" containerName="main"
Jul 08 04:04:27.660599 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660343    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660599 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660355    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5186754-6d56-477a-b986-12ca41894817" containerName="istio-proxy"
Jul 08 04:04:27.660599 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.660363    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="36ea6388-0dbc-497b-aef6-8baff82d8819" containerName="main"
Jul 08 04:04:27.663706 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.663682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.666050 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.666019    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvjg8\"/\"kube-root-ca.crt\""
Jul 08 04:04:27.666204 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.666035    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-hvjg8\"/\"openshift-service-ca.crt\""
Jul 08 04:04:27.666540 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.666524    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-hvjg8\"/\"default-dockercfg-w6npk\""
Jul 08 04:04:27.672566 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.672539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/must-gather-4kxtx"]
Jul 08 04:04:27.840769 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.840733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab28aefa-6794-4780-9c93-abce2e911eff-must-gather-output\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.840963 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.840814    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-459fd\" (UniqueName: \"kubernetes.io/projected/ab28aefa-6794-4780-9c93-abce2e911eff-kube-api-access-459fd\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.942110 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.942020    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab28aefa-6794-4780-9c93-abce2e911eff-must-gather-output\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.942110 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.942080    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-459fd\" (UniqueName: \"kubernetes.io/projected/ab28aefa-6794-4780-9c93-abce2e911eff-kube-api-access-459fd\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.942397 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.942376    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ab28aefa-6794-4780-9c93-abce2e911eff-must-gather-output\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.949822 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.949790    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-459fd\" (UniqueName: \"kubernetes.io/projected/ab28aefa-6794-4780-9c93-abce2e911eff-kube-api-access-459fd\") pod \"must-gather-4kxtx\" (UID: \"ab28aefa-6794-4780-9c93-abce2e911eff\") " pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:27.974909 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:27.974867    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/must-gather-4kxtx"
Jul 08 04:04:28.112043 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:28.112017    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/must-gather-4kxtx"]
Jul 08 04:04:28.113352 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:04:28.113324    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab28aefa_6794_4780_9c93_abce2e911eff.slice/crio-5405c2006c44f9f0a2811f61d6bfee7ed5b630946ca8472c456e12d4a0a8408b WatchSource:0}: Error finding container 5405c2006c44f9f0a2811f61d6bfee7ed5b630946ca8472c456e12d4a0a8408b: Status 404 returned error can't find the container with id 5405c2006c44f9f0a2811f61d6bfee7ed5b630946ca8472c456e12d4a0a8408b
Jul 08 04:04:28.115367 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:28.115351    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 04:04:28.440465 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:28.440429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/must-gather-4kxtx" event={"ID":"ab28aefa-6794-4780-9c93-abce2e911eff","Type":"ContainerStarted","Data":"5405c2006c44f9f0a2811f61d6bfee7ed5b630946ca8472c456e12d4a0a8408b"}
Jul 08 04:04:29.450710 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:29.450617    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/must-gather-4kxtx" event={"ID":"ab28aefa-6794-4780-9c93-abce2e911eff","Type":"ContainerStarted","Data":"774595a5c37f5c75a48296dc3323f4e460a4d4f7070712df789dc19341cad13b"}
Jul 08 04:04:29.450710 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:29.450673    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/must-gather-4kxtx" event={"ID":"ab28aefa-6794-4780-9c93-abce2e911eff","Type":"ContainerStarted","Data":"0859689d8de0aa8f38d87cfdaaf4baf3d3fc4937991c54d5f605ffc6494d3b2b"}
Jul 08 04:04:29.466411 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:29.466341    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hvjg8/must-gather-4kxtx" podStartSLOduration=1.444015734 podStartE2EDuration="2.466318848s" totalImagesPullingTime="1.022303114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 04:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:04:29.465236734 +0000 UTC m=+2680.573582700" watchObservedRunningTime="2026-07-08 04:04:29.466318848 +0000 UTC m=+2680.574664789"
Jul 08 04:04:30.784364 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:30.784334    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h6zkr_496ab302-a93d-4c2b-96ba-7632f4c92c9c/global-pull-secret-syncer/0.log"
Jul 08 04:04:30.923734 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:30.923698    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jgnfd_969cb854-5ab3-4f0a-908d-762dc8b0955a/konnectivity-agent/0.log"
Jul 08 04:04:31.003529 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:31.003494    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-160.ec2.internal_4c4d2eab85087dd7f7d7ed97e5b0c0e8/haproxy/0.log"
Jul 08 04:04:35.493272 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.493216    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/extract/0.log"
Jul 08 04:04:35.524765 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.524733    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/util/0.log"
Jul 08 04:04:35.555103 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.555075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759z7vth_b94ee3b1-440d-4b24-bcfd-66b413ce3ce1/pull/0.log"
Jul 08 04:04:35.588119 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.588075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/extract/0.log"
Jul 08 04:04:35.615034 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.614995    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/util/0.log"
Jul 08 04:04:35.638743 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.638709    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04gvbp_f3249e46-b90b-4afd-be17-2718d25b1a7f/pull/0.log"
Jul 08 04:04:35.665437 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.665405    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/extract/0.log"
Jul 08 04:04:35.693340 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.693297    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/util/0.log"
Jul 08 04:04:35.722718 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.722683    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73jswfn_960f06b0-4537-4872-9133-f79d69b51320/pull/0.log"
Jul 08 04:04:35.769349 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.769177    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/extract/0.log"
Jul 08 04:04:35.795957 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.795931    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/util/0.log"
Jul 08 04:04:35.816894 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:35.816859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ftsbj_8039744b-ec8b-4c9e-854b-b62b5ef5134d/pull/0.log"
Jul 08 04:04:36.284172 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.283980    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76b46ff5d4-rk57q_f077e16d-a017-48e7-a15c-f8c1420ef50b/authorino/0.log"
Jul 08 04:04:36.333418 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.333382    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-z9p6p_d62ec88a-5e14-4243-836b-8b0e6f70daff/manager/0.log"
Jul 08 04:04:36.361889 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.361855    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-95jls_fdf922e1-1caa-41c6-9f2a-2817899411a0/manager/0.log"
Jul 08 04:04:36.432960 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.432918    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2mnhl_8e852069-5b7e-46aa-9171-d3cd658d5a40/registry-server/0.log"
Jul 08 04:04:36.664165 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.663455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-9z84k_8d650c5b-7758-45a7-bbe6-6b2ebeabd132/manager/0.log"
Jul 08 04:04:36.778002 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:36.777960    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-ztm2h_cb870b59-f919-45ce-9977-38d8ca77dfaf/manager/0.log"
Jul 08 04:04:38.563542 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.563457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/alertmanager/0.log"
Jul 08 04:04:38.588169 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.588110    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/config-reloader/0.log"
Jul 08 04:04:38.609475 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.609443    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/kube-rbac-proxy-web/0.log"
Jul 08 04:04:38.631999 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.631969    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/kube-rbac-proxy/0.log"
Jul 08 04:04:38.657008 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.656967    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/kube-rbac-proxy-metric/0.log"
Jul 08 04:04:38.689571 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.689547    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/prom-label-proxy/0.log"
Jul 08 04:04:38.712510 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.712484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_421626d6-5819-4a78-9555-8dbff83165c9/init-config-reloader/0.log"
Jul 08 04:04:38.791991 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.791959    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ndjwj_0d972215-36e6-46db-a1ae-5bd18cb5b4aa/kube-state-metrics/0.log"
Jul 08 04:04:38.810097 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.810068    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ndjwj_0d972215-36e6-46db-a1ae-5bd18cb5b4aa/kube-rbac-proxy-main/0.log"
Jul 08 04:04:38.832608 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:38.832514    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-ndjwj_0d972215-36e6-46db-a1ae-5bd18cb5b4aa/kube-rbac-proxy-self/0.log"
Jul 08 04:04:39.150473 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:39.150336    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl8vm_f2d3735b-eea5-4dd0-a6b0-13cc4e84894d/node-exporter/0.log"
Jul 08 04:04:39.176284 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:39.176254    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl8vm_f2d3735b-eea5-4dd0-a6b0-13cc4e84894d/kube-rbac-proxy/0.log"
Jul 08 04:04:39.198461 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:39.198435    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-wl8vm_f2d3735b-eea5-4dd0-a6b0-13cc4e84894d/init-textfile/0.log"
Jul 08 04:04:41.266402 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.266364    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"]
Jul 08 04:04:41.274434 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.274399    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.278960 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.278930    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"]
Jul 08 04:04:41.296980 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.296926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8q89\" (UniqueName: \"kubernetes.io/projected/ac7c9593-75a2-4f6d-ab98-fac649b12099-kube-api-access-w8q89\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.297288 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.297267    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-proc\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.297610 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.297590    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-podres\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.297813 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.297797    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-sys\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.297971 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.297954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-lib-modules\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.399293 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.399257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-podres\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.399562 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.399542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-sys\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.399698 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.399682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-lib-modules\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.399880 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.399865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8q89\" (UniqueName: \"kubernetes.io/projected/ac7c9593-75a2-4f6d-ab98-fac649b12099-kube-api-access-w8q89\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.399994 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.399981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-proc\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.400418 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.400382    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-podres\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.400548 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.400517    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-lib-modules\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.400620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.400559    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-proc\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.400620 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.400577    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ac7c9593-75a2-4f6d-ab98-fac649b12099-sys\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.408776 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.408721    2563 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8q89\" (UniqueName: \"kubernetes.io/projected/ac7c9593-75a2-4f6d-ab98-fac649b12099-kube-api-access-w8q89\") pod \"perf-node-gather-daemonset-52rg5\" (UID: \"ac7c9593-75a2-4f6d-ab98-fac649b12099\") " pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.570396 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.570369    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 04:04:41.579261 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.579227    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/2.log"
Jul 08 04:04:41.603088 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.603057    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:41.797763 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:41.797735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"]
Jul 08 04:04:41.799552 ip-10-0-137-160 kubenswrapper[2563]: W0708 04:04:41.799522    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podac7c9593_75a2_4f6d_ab98_fac649b12099.slice/crio-a298d70828ff67d14a1d07914ffc0c7005849adc9c7a39d3f9027a075d698a2d WatchSource:0}: Error finding container a298d70828ff67d14a1d07914ffc0c7005849adc9c7a39d3f9027a075d698a2d: Status 404 returned error can't find the container with id a298d70828ff67d14a1d07914ffc0c7005849adc9c7a39d3f9027a075d698a2d
Jul 08 04:04:42.076582 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:42.076496    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7874684746-58smt_4f024b8b-29db-48fd-93a7-6681184253e0/console/0.log"
Jul 08 04:04:42.598836 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:42.598792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5" event={"ID":"ac7c9593-75a2-4f6d-ab98-fac649b12099","Type":"ContainerStarted","Data":"6cabc3898bd75390c8a1daff0de549c4a20fc18790d791f2907f35ceda472163"}
Jul 08 04:04:42.599333 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:42.598909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5" event={"ID":"ac7c9593-75a2-4f6d-ab98-fac649b12099","Type":"ContainerStarted","Data":"a298d70828ff67d14a1d07914ffc0c7005849adc9c7a39d3f9027a075d698a2d"}
Jul 08 04:04:42.599333 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:42.598939    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:42.617161 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:42.616346    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5" podStartSLOduration=1.6163272709999998 podStartE2EDuration="1.616327271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 04:04:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 04:04:42.612971719 +0000 UTC m=+2693.721317661" watchObservedRunningTime="2026-07-08 04:04:42.616327271 +0000 UTC m=+2693.724673212"
Jul 08 04:04:43.758483 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:43.758455    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9cxdf_a3d498c1-68f2-4915-8504-80e20964154d/dns/0.log"
Jul 08 04:04:43.780517 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:43.780484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9cxdf_a3d498c1-68f2-4915-8504-80e20964154d/kube-rbac-proxy/0.log"
Jul 08 04:04:43.898727 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:43.898694    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ffj7r_9478073e-2c45-4a5f-838f-ae62733ea6ee/dns-node-resolver/0.log"
Jul 08 04:04:44.519857 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:44.519814    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hpqlc_33643870-0989-4f93-9e49-61cd14adfd7f/node-ca/0.log"
Jul 08 04:04:45.616070 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:45.616029    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-b7drh_2612f803-f607-4d02-bc14-0ba3328c8d03/kube-auth-proxy/0.log"
Jul 08 04:04:45.814487 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:45.814457    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6rfwn_e480dfee-6196-42e2-abc0-9e0b027f4db5/payload-processing/0.log"
Jul 08 04:04:45.837506 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:45.837477    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cf9c66499-8vtcz_aff7f08a-06ac-41bb-9e76-14f8670b4c84/router/0.log"
Jul 08 04:04:46.544724 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:46.544697    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mbkbh_d19979df-55c8-4e1e-8a25-0390633d1794/serve-healthcheck-canary/0.log"
Jul 08 04:04:47.030590 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:47.030502    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-r6rvp_2f8cdf2b-e0ba-4223-aa42-431ec6ac0814/insights-operator/0.log"
Jul 08 04:04:47.033066 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:47.033045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-r6rvp_2f8cdf2b-e0ba-4223-aa42-431ec6ac0814/insights-operator/1.log"
Jul 08 04:04:47.195051 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:47.195012    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfqgx_047a2caa-fd23-46fb-9fbe-91cadb1a5788/kube-rbac-proxy/0.log"
Jul 08 04:04:47.215401 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:47.215368    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfqgx_047a2caa-fd23-46fb-9fbe-91cadb1a5788/exporter/0.log"
Jul 08 04:04:47.236950 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:47.236913    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mfqgx_047a2caa-fd23-46fb-9fbe-91cadb1a5788/extractor/0.log"
Jul 08 04:04:48.617522 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:48.617492    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-hvjg8/perf-node-gather-daemonset-52rg5"
Jul 08 04:04:49.326317 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.326251    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-zdd8t_45057a34-5fa4-44bf-bfa3-68afa7748696/manager/0.log"
Jul 08 04:04:49.577319 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.577233    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-rr99j_c4f1f2e3-aa65-48e1-ad25-c2f301658ed1/manager/0.log"
Jul 08 04:04:49.628490 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.628454    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-bcbd5b4c9-2dd2c_d752bb66-9c4f-4ea7-ae8f-31af63170554/maas-api/0.log"
Jul 08 04:04:49.650866 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.650828    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29724705-sjrf8_58723121-7b6d-4fbb-9ac8-eee93d38dfb9/cleanup/0.log"
Jul 08 04:04:49.841124 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.841045    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-b857d6d59-k95zk_632d8a6d-259b-4780-9571-3aa34bcfe8f5/manager/0.log"
Jul 08 04:04:49.863275 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.863249    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-vwb5b_583ae9b9-5b3b-4ccc-b932-9ddcd9daa3fa/server/0.log"
Jul 08 04:04:49.970754 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:49.970706    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bl9cn_115aced2-45b7-4e4b-9ddb-4424117fac5b/manager/0.log"
Jul 08 04:04:50.144294 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.144207    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c_c87e613a-f280-4a78-8f4b-fd8579bd264e/manager/0.log"
Jul 08 04:04:50.167502 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.167473    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-bfdfcb4c5-lxx9c_c87e613a-f280-4a78-8f4b-fd8579bd264e/copy-manifests/0.log"
Jul 08 04:04:50.195668 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.195639    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-2j4bv_53cbffe1-3a17-4d0b-b882-87ef081af3fd/postgres/0.log"
Jul 08 04:04:50.244356 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.243666    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 04:04:50.261162 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.259075    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-9lstn_788ca259-379f-4dfb-9f16-9fc1f443ecd8/console-operator/1.log"
Jul 08 04:04:50.261162 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.259199    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 04:04:50.270035 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:50.270003    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-h5lx7_407827ad-6c24-4b50-80b2-a488d63f499f/ovn-acl-logging/0.log"
Jul 08 04:04:51.799124 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:51.799088    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-vwgnr_8696c5c5-39ad-4b62-94d4-fe8967a672e2/openshift-lws-operator/0.log"
Jul 08 04:04:52.287409 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:52.287313    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-b7drh_2612f803-f607-4d02-bc14-0ba3328c8d03/kube-auth-proxy/0.log"
Jul 08 04:04:52.454690 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:52.454654    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6rfwn_e480dfee-6196-42e2-abc0-9e0b027f4db5/payload-processing/0.log"
Jul 08 04:04:52.468286 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:52.468254    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cf9c66499-8vtcz_aff7f08a-06ac-41bb-9e76-14f8670b4c84/router/0.log"
Jul 08 04:04:53.836718 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:53.836687    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69556cfdfb-b7drh_2612f803-f607-4d02-bc14-0ba3328c8d03/kube-auth-proxy/0.log"
Jul 08 04:04:53.995770 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:53.995742    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-6rfwn_e480dfee-6196-42e2-abc0-9e0b027f4db5/payload-processing/0.log"
Jul 08 04:04:54.017823 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:54.017793    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-cf9c66499-8vtcz_aff7f08a-06ac-41bb-9e76-14f8670b4c84/router/0.log"
Jul 08 04:04:58.297815 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.297781    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-47nlr_77286f6d-68a7-41d3-98d2-dd0201a623ea/kube-multus/0.log"
Jul 08 04:04:58.643268 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.643185    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/kube-multus-additional-cni-plugins/0.log"
Jul 08 04:04:58.663442 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.663413    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/egress-router-binary-copy/0.log"
Jul 08 04:04:58.683412 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.683385    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/cni-plugins/0.log"
Jul 08 04:04:58.706749 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.706725    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/bond-cni-plugin/0.log"
Jul 08 04:04:58.727301 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.727276    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/routeoverride-cni/0.log"
Jul 08 04:04:58.747029 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.746999    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/whereabouts-cni-bincopy/0.log"
Jul 08 04:04:58.766989 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.766958    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wf9lr_ba1ffb6a-2588-4411-b936-433be263812f/whereabouts-cni/0.log"
Jul 08 04:04:58.913205 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.913089    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v5fdt_848ea393-51f5-40f4-919f-7b147f20ff22/network-metrics-daemon/0.log"
Jul 08 04:04:58.935247 ip-10-0-137-160 kubenswrapper[2563]: I0708 04:04:58.935213    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-v5fdt_848ea393-51f5-40f4-919f-7b147f20ff22/kube-rbac-proxy/0.log"