Jul 14 17:52:26.370551 ip-10-0-130-101 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 14 17:52:26.370562 ip-10-0-130-101 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 14 17:52:26.370570 ip-10-0-130-101 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 14 17:52:26.370878 ip-10-0-130-101 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 14 17:52:36.555689 ip-10-0-130-101 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 14 17:52:36.555709 ip-10-0-130-101 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot eef407ea63e54b6aa441c676784884e1 --
Jul 14 17:54:41.684851 ip-10-0-130-101 systemd[1]: Starting Kubernetes Kubelet...
Jul 14 17:54:42.056741 ip-10-0-130-101 kubenswrapper[2556]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:54:42.056741 ip-10-0-130-101 kubenswrapper[2556]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 14 17:54:42.056741 ip-10-0-130-101 kubenswrapper[2556]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:54:42.056741 ip-10-0-130-101 kubenswrapper[2556]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 14 17:54:42.056741 ip-10-0-130-101 kubenswrapper[2556]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 14 17:54:42.058628 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.058551    2556 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 14 17:54:42.064862 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064847    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.064862 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064862    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064866    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064869    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064872    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064876    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064879    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064882    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064885    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064887    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064890    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064893    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064896    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064898    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064901    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064904    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064907    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064909    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064911    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064914    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064916    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.064922 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064919    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064921    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064925    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064927    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064930    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064933    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064936    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064939    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064941    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064944    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064947    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064949    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064951    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064955    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064959    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064962    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064964    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064967    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064970    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.065374 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064972    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064974    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064977    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064979    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064982    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064984    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064989    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064993    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064997    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.064999    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065002    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065005    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065008    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065010    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065013    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065016    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065019    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065021    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.065833 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065024    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065026    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065029    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065031    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065033    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065036    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065038    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065041    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065043    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065045    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065048    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065050    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065053    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065056    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065058    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065061    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065063    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065065    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065068    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065070    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.066240 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065073    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065075    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065078    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065080    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065082    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065085    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065087    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065089    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065092    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065095    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065097    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065100    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065102    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065105    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065107    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065110    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065112    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065114    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065117    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065119    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.066711 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065121    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065806    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065814    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065817    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065819    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065822    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065825    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065828    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065830    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065833    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065835    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065838    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065840    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065843    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065845    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065848    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065850    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065853    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065855    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065858    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.067194 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065860    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065865    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065869    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065872    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065875    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065878    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065881    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065883    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065886    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065889    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065892    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065894    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065897    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065899    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065904    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065907    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065910    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065912    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065915    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.067672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065918    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065921    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065923    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065925    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065928    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065930    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065933    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065935    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065937    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065941    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065943    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065945    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065948    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065951    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065954    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065956    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065959    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065961    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065963    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.068129 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065966    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065968    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065970    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065973    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065975    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065977    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065980    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065984    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065987    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065989    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065992    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065994    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065996    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.065999    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066002    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066004    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066010    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066013    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066016    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066018    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.068575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066021    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066024    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066026    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066029    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066031    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066034    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066036    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066039    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066041    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066044    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066046    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066048    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066051    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066053    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066055    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066058    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066060    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066063    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066065    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066067    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.069049 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066070    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066072    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066415    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066420    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066422    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066425    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066433    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066436    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066438    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066441    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066443    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066446    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066449    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066451    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066454    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066457    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066460    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066462    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066465    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.069524 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066467    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066470    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066472    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066474    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066477    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066479    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066481    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066484    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066488    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066492    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066494    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066497    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066500    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066503    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066506    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066509    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066512    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066514    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066517    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.069984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066519    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066522    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066525    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066527    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066530    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066532    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066534    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066537    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066539    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066542    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066545    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066548    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066550    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066552    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066555    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066557    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066560    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066563    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066565    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066567    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.070418 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066570    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066572    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066574    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066577    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066596    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066599    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066601    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066604    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066606    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066609    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066612    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066614    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066617    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066619    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066623    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066625    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066627    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066630    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066632    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066635    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066641    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.070900 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066643    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066647    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066650    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066653    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066656    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066658    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066661    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066663    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066666    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066668    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066671    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066673    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066675    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066678    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066681    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066683    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066686    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066688    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066692    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.071391 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066695    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066698    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.066700    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066766    2556 flags.go:64] FLAG: --address="0.0.0.0"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066773    2556 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066780    2556 flags.go:64] FLAG: --anonymous-auth="true"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066784    2556 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066788    2556 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066791    2556 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066795    2556 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066799    2556 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066803    2556 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066806    2556 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066810    2556 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066813    2556 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066816    2556 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066819    2556 flags.go:64] FLAG: --cgroup-root=""
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066821    2556 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066824    2556 flags.go:64] FLAG: --client-ca-file=""
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066827    2556 flags.go:64] FLAG: --cloud-provider="external"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066829    2556 flags.go:64] FLAG: --cluster-dns="[]"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066833    2556 flags.go:64] FLAG: --cluster-domain=""
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066835    2556 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066838    2556 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 14 17:54:42.071878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066841    2556 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066844    2556 flags.go:64] FLAG: --container-log-max-files="5"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066847    2556 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066850    2556 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066853    2556 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066856    2556 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066858    2556 flags.go:64] FLAG: --contention-profiling="false"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066861    2556 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066865    2556 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066868    2556 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066871    2556 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066874    2556 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066877    2556 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066880    2556 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066883    2556 flags.go:64] FLAG: --enable-load-reader="false"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066885    2556 flags.go:64] FLAG: --enable-server="true"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066888    2556 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066893    2556 flags.go:64] FLAG: --event-burst="100"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066896    2556 flags.go:64] FLAG: --event-qps="50"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066898    2556 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066902    2556 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066905    2556 flags.go:64] FLAG: --eviction-hard=""
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066908    2556 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066911    2556 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066913    2556 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 14 17:54:42.072435 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066916    2556 flags.go:64] FLAG: --eviction-soft=""
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066919    2556 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066921    2556 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066924    2556 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066927    2556 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066929    2556 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066932    2556 flags.go:64] FLAG: --fail-swap-on="true"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066934    2556 flags.go:64] FLAG: --feature-gates=""
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066938    2556 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066941    2556 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066944    2556 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066946    2556 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066949    2556 flags.go:64] FLAG: --healthz-port="10248"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066952    2556 flags.go:64] FLAG: --help="false"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066954    2556 flags.go:64] FLAG: --hostname-override="ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066959    2556 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066961    2556 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066964    2556 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066967    2556 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066970    2556 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066973    2556 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066975    2556 flags.go:64] FLAG: --image-service-endpoint=""
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066978    2556 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 14 17:54:42.073050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066981    2556 flags.go:64] FLAG: --kube-api-burst="100"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066984    2556 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066987    2556 flags.go:64] FLAG: --kube-api-qps="50"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066989    2556 flags.go:64] FLAG: --kube-reserved=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066993    2556 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066995    2556 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.066998    2556 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067001    2556 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067003    2556 flags.go:64] FLAG: --lock-file=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067006    2556 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067009    2556 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067012    2556 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067016    2556 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067019    2556 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067022    2556 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067024    2556 flags.go:64] FLAG: --logging-format="text"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067027    2556 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067030    2556 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067032    2556 flags.go:64] FLAG: --manifest-url=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067035    2556 flags.go:64] FLAG: --manifest-url-header=""
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067039    2556 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067042    2556 flags.go:64] FLAG: --max-open-files="1000000"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067045    2556 flags.go:64] FLAG: --max-pods="110"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067048    2556 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067052    2556 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 14 17:54:42.073651 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067055    2556 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067058    2556 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067061    2556 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067063    2556 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067066    2556 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067072    2556 flags.go:64] FLAG: --node-status-max-images="50"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067075    2556 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067078    2556 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067081    2556 flags.go:64] FLAG: --pod-cidr=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067084    2556 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067089    2556 flags.go:64] FLAG: --pod-manifest-path=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067091    2556 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067095    2556 flags.go:64] FLAG: --pods-per-core="0"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067098    2556 flags.go:64] FLAG: --port="10250"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067101    2556 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067103    2556 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02fbc41601f24fcdb"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067106    2556 flags.go:64] FLAG: --qos-reserved=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067109    2556 flags.go:64] FLAG: --read-only-port="10255"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067111    2556 flags.go:64] FLAG: --register-node="true"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067114    2556 flags.go:64] FLAG: --register-with-taints=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067121    2556 flags.go:64] FLAG: --registry-burst="10"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067124    2556 flags.go:64] FLAG: --registry-qps="5"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067127    2556 flags.go:64] FLAG: --reserved-cpus=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067129    2556 flags.go:64] FLAG: --reserved-memory=""
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067133    2556 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 14 17:54:42.074226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067135    2556 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067138    2556 flags.go:64] FLAG: --rotate-certificates="false"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067141    2556 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067143    2556 flags.go:64] FLAG: --runonce="false"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067146    2556 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067149    2556 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067153    2556 flags.go:64] FLAG: --seccomp-default="false"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067155    2556 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067158    2556 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067161    2556 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067163    2556 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067166    2556 flags.go:64] FLAG: --storage-driver-password="root"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067169    2556 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067171    2556 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067174    2556 flags.go:64] FLAG: --storage-driver-user="root"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067176    2556 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067179    2556 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067182    2556 flags.go:64] FLAG: --system-cgroups=""
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067185    2556 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067191    2556 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067194    2556 flags.go:64] FLAG: --tls-cert-file=""
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067196    2556 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067200    2556 flags.go:64] FLAG: --tls-min-version=""
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067203    2556 flags.go:64] FLAG: --tls-private-key-file=""
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067206    2556 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 14 17:54:42.074855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067209    2556 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067211    2556 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067214    2556 flags.go:64] FLAG: --v="2"
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067218    2556 flags.go:64] FLAG: --version="false"
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067221    2556 flags.go:64] FLAG: --vmodule=""
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067224    2556 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.067227    2556 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067333    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067336    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067339    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067341    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067344    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067346    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067350    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067353    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067355    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067358    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067360    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067362    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067365    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067367    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.075450 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067370    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067372    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067375    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067377    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067381    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067384    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067387    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067389    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067391    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067394    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067396    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067398    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067401    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067403    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067406    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067408    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067410    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067413    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067415    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.075955 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067419    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067422    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067424    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067427    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067429    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067433    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067435    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067437    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067440    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067443    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067447    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067449    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067452    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067454    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067457    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067460    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067462    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067464    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067467    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.076596 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067470    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067472    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067475    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067477    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067479    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067482    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067484    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067486    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067489    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067491    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067493    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067496    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067498    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067500    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067503    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067505    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067507    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067510    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067513    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067515    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.077272 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067518    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067520    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067523    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067525    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067527    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067530    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067532    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067535    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067537    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067540    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067542    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067544    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067547    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067549    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067552    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067554    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067556    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067559    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067561    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067563    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.077766 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067566    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067568    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067570    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067573    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067576    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067593    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.067596    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.068348    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.074817    2556 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.074834    2556 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.075651    2556 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.075658    2556 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075730    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075735    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.078241 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075738    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075741    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075744    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075747    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075750    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075752    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075755    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075758    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075761    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075764    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075766    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075769    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075772    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075775    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075777    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075780    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075782    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075785    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075788    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075790    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.078577 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075793    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075795    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075799    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075803    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075806    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075810    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075813    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075821    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075824    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075826    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075828    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075831    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075833    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075836    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075838    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075840    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075844    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075847    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075849    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.079048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075851    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075854    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075856    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075859    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075861    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075864    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075866    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075868    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075871    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075874    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075877    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075880    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075883    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075885    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075887    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075889    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075892    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075894    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075897    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.079485 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075899    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075902    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075909    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075912    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075914    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075917    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075919    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075922    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075924    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075926    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075929    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075931    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075933    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075936    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075938    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075941    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075944    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075946    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075948    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075950    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.079984 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075953    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075955    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075957    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075960    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075962    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075965    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075967    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075970    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075972    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075975    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075977    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075979    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075981    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075984    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075986    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075993    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075996    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.075998    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.080442 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.076001    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.076006    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077921    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077937    2556 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077942    2556 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077947    2556 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077950    2556 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077954    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077958    2556 feature_gate.go:328] unrecognized feature gate: Example2
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077962    2556 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077966    2556 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077970    2556 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077974    2556 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077977    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077981    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 14 17:54:42.080887 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077985    2556 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077989    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077992    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.077997    2556 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078001    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078005    2556 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078011    2556 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078016    2556 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078021    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078025    2556 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078029    2556 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078032    2556 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078036    2556 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078041    2556 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078049    2556 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078053    2556 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078057    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078061    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078065    2556 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 14 17:54:42.081235 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078069    2556 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078073    2556 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078076    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078080    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078084    2556 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078087    2556 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078091    2556 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078095    2556 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078099    2556 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078104    2556 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078108    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078111    2556 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078115    2556 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078119    2556 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078122    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078124    2556 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078127    2556 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078129    2556 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078132    2556 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078134    2556 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 14 17:54:42.081684 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078137    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078139    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078141    2556 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078144    2556 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078146    2556 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078149    2556 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078151    2556 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078154    2556 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078156    2556 feature_gate.go:328] unrecognized feature gate: Example
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078158    2556 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078161    2556 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078163    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078165    2556 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078169    2556 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078173    2556 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078176    2556 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078178    2556 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078181    2556 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078185    2556 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 14 17:54:42.082133 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078188    2556 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078190    2556 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078193    2556 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078195    2556 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078198    2556 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078201    2556 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078203    2556 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078205    2556 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078207    2556 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078210    2556 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078212    2556 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078215    2556 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078218    2556 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078221    2556 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078223    2556 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078225    2556 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078228    2556 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078230    2556 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078232    2556 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078234    2556 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 14 17:54:42.082574 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078237    2556 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078239    2556 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078243    2556 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078245    2556 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078248    2556 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078250    2556 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078253    2556 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:42.078256    2556 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.078261    2556 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.078356    2556 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.080324    2556 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.081415    2556 server.go:1019] "Starting client certificate rotation"
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.081512    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 17:54:42.083042 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.081536    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 14 17:54:42.083680 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.083667    2556 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 17:54:42.085778 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.085756    2556 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 14 17:54:42.096758 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.096741    2556 log.go:25] "Validated CRI v1 runtime API"
Jul 14 17:54:42.098974 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.098956    2556 log.go:25] "Validated CRI v1 image API"
Jul 14 17:54:42.099844 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.099830    2556 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 14 17:54:42.102134 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.102116    2556 fs.go:135] Filesystem UUIDs: map[1bbb4891-8f16-427b-85fa-4f8d56cb3e9d:/dev/nvme0n1p3 540f34e1-2da2-43b2-bc85-8ce63121b8d1:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 14 17:54:42.102217 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.102132    2556 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 14 17:54:42.107853 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.107755    2556 manager.go:216] Machine: {Timestamp:2026-07-14 17:54:42.106555302 +0000 UTC m=+0.308110433 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3192625 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2f74917be1c45e9a7224d501063879 SystemUUID:ec2f7491-7be1-c45e-9a72-24d501063879 BootID:eef407ea-63e5-4b6a-a441-c676784884e1 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:78:9c:c7:91:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:78:9c:c7:91:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:f3:13:5d:ba:82 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 14 17:54:42.107853 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.107847    2556 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 14 17:54:42.107951 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.107910    2556 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 14 17:54:42.108961 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.108941    2556 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 14 17:54:42.109086 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.108969    2556 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 14 17:54:42.110333 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.110322    2556 topology_manager.go:138] "Creating topology manager with none policy"
Jul 14 17:54:42.110374 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.110336    2556 container_manager_linux.go:309] "Creating device plugin manager"
Jul 14 17:54:42.110374 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.110343    2556 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 17:54:42.111028 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.111018    2556 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 14 17:54:42.112020 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.112011    2556 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 14 17:54:42.115214 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.115203    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 17:54:42.115324 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.115315    2556 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 14 17:54:42.115981 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.115973    2556 kubelet.go:486] "Attempting to sync node with API server"
Jul 14 17:54:42.116011 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.115990    2556 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 14 17:54:42.116011 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.116004    2556 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 14 17:54:42.116011 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.116012    2556 kubelet.go:398] "Adding apiserver pod source"
Jul 14 17:54:42.116107 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.116025    2556 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 14 17:54:42.116340 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.116329    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 17:54:42.116387 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.116348    2556 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 14 17:54:42.118484 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.118468    2556 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 14 17:54:42.120691 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.120670    2556 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 14 17:54:42.120763 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.120717    2556 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 14 17:54:42.122674 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122660    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122685    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122692    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122697    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122705    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122710    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122718    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122723    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122729    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 14 17:54:42.122747 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122734    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 14 17:54:42.123019 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122753    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 14 17:54:42.123019 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122761    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 14 17:54:42.123019 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.122922    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 17:54:42.123423 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.123412    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 14 17:54:42.123423 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.123423    2556 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 14 17:54:42.126210 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.126185    2556 server.go:1268] "Started kubelet"
Jul 14 17:54:42.126284 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.126252    2556 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 14 17:54:42.126337 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.126257    2556 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 14 17:54:42.126337 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.126320    2556 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 14 17:54:42.126990 ip-10-0-130-101 systemd[1]: Started Kubernetes Kubelet.
Jul 14 17:54:42.127926 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.127908    2556 server.go:310] "Adding debug handlers to kubelet server"
Jul 14 17:54:42.128152 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.128132    2556 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 14 17:54:42.133048 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133030    2556 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 14 17:54:42.133165 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.133142    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.133165 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133036    2556 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 14 17:54:42.133848 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133825    2556 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 14 17:54:42.133848 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133827    2556 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 14 17:54:42.133987 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133825    2556 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 14 17:54:42.133987 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133870    2556 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 14 17:54:42.133987 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.133911    2556 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 14 17:54:42.133987 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133974    2556 reconstruct.go:97] "Volume reconstruction finished"
Jul 14 17:54:42.133987 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.133984    2556 reconciler.go:29] "Reconciler: start to sync state"
Jul 14 17:54:42.134194 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.134033    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 14 17:54:42.134194 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.134074    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 14 17:54:42.134325 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134228    2556 factory.go:55] Registering systemd factory
Jul 14 17:54:42.134325 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134259    2556 factory.go:223] Registration of the systemd container factory successfully
Jul 14 17:54:42.134440 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134429    2556 factory.go:153] Registering CRI-O factory
Jul 14 17:54:42.134500 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134444    2556 factory.go:223] Registration of the crio container factory successfully
Jul 14 17:54:42.134500 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134492    2556 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 14 17:54:42.134608 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134515    2556 factory.go:103] Registering Raw factory
Jul 14 17:54:42.134608 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134531    2556 manager.go:1196] Started watching for new ooms in manager
Jul 14 17:54:42.134608 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.133687    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d7611625f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.126160479 +0000 UTC m=+0.327715611,LastTimestamp:2026-07-14 17:54:42.126160479 +0000 UTC m=+0.327715611,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.134915 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.134897    2556 manager.go:349] Starting recovery of all containers
Jul 14 17:54:42.141725 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.141602    2556 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 14 17:54:42.141825 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.141712    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 14 17:54:42.146422 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.146400    2556 manager.go:354] Recovery completed
Jul 14 17:54:42.148832 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.148797    2556 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 14 17:54:42.152134 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.152121    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.154294 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154279    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.154357 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154306    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.154357 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154315    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.154876 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154862    2556 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 14 17:54:42.154876 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154875    2556 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 14 17:54:42.154958 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154889    2556 state_mem.go:36] "Initialized new in-memory state store"
Jul 14 17:54:42.154958 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.154922    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.155629 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.155611    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.155701 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.155643    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.155701 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.155655    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.156173 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.156158    2556 policy_none.go:49] "None policy: Start"
Jul 14 17:54:42.156207 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.156176    2556 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 14 17:54:42.156207 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.156186    2556 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 14 17:54:42.156696 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.156629    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77beaa96  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.15429391 +0000 UTC m=+0.355849040,LastTimestamp:2026-07-14 17:54:42.15429391 +0000 UTC m=+0.355849040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.157138 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.157126    2556 policy_none.go:47] "Start"
Jul 14 17:54:42.167990 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.167928    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77beecd5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.154310869 +0000 UTC m=+0.355866000,LastTimestamp:2026-07-14 17:54:42.154310869 +0000 UTC m=+0.355866000,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.175279 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.175220    2556 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77bf0db0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.15431928 +0000 UTC m=+0.355874412,LastTimestamp:2026-07-14 17:54:42.15431928 +0000 UTC m=+0.355874412,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.178526 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.178468    2556 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-101.ec2.internal.18c2398d77beaa96\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77beaa96  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.15429391 +0000 UTC m=+0.355849040,LastTimestamp:2026-07-14 17:54:42.155628055 +0000 UTC m=+0.357183186,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.188049 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.187996    2556 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-101.ec2.internal.18c2398d77beecd5\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77beecd5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.154310869 +0000 UTC m=+0.355866000,LastTimestamp:2026-07-14 17:54:42.155649376 +0000 UTC m=+0.357204506,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.191978 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.191962    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dpzq5"
Jul 14 17:54:42.195125 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195115    2556 manager.go:337] "Starting Device Plugin manager"
Jul 14 17:54:42.195181 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.195140    2556 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 14 17:54:42.195181 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195149    2556 server.go:87] "Starting device plugin registration server"
Jul 14 17:54:42.195343 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195328    2556 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 14 17:54:42.195402 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195345    2556 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 14 17:54:42.195447 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195429    2556 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 14 17:54:42.195535 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195520    2556 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 14 17:54:42.195535 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.195533    2556 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 14 17:54:42.196015 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.195998    2556 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 14 17:54:42.196078 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.196045    2556 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.203125 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.203109    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dpzq5"
Jul 14 17:54:42.204843 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.204791    2556 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-130-101.ec2.internal.18c2398d77bf0db0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-101.ec2.internal.18c2398d77bf0db0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-101.ec2.internal,UID:ip-10-0-130-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-130-101.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-130-101.ec2.internal,},FirstTimestamp:2026-07-14 17:54:42.15431928 +0000 UTC m=+0.355874412,LastTimestamp:2026-07-14 17:54:42.155658349 +0000 UTC m=+0.357213480,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-101.ec2.internal,}"
Jul 14 17:54:42.295996 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.295972    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.297091 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.296896    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.297091 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.296924    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.297091 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.296936    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.297091 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.296960    2556 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.304778 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.304766    2556 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.304819 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.304785    2556 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-101.ec2.internal\": node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.311839 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.311792    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 14 17:54:42.313147 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.313130    2556 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 14 17:54:42.313221 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.313164    2556 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 14 17:54:42.313221 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.313185    2556 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 14 17:54:42.313221 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.313213    2556 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 14 17:54:42.315281 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.315258    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:54:42.319559 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.319544    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.413986 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.413966    2556 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"]
Jul 14 17:54:42.414036 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.414028    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.414706 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.414693    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.414772 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.414717    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.414772 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.414726    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.416108 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416089    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.416234 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416220    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.416287 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416253    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.416720 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416701    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.416799 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416728    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.416799 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416738    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.416799 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416702    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.416799 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416795    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.416946 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.416806    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.418117 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.418101    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.418205 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.418124    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:42.419426 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.419411    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:42.419511 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.419439    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:42.419511 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.419450    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:42.419637 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.419624    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.435953 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.435936    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.436007 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.435960    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.436007 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.435984    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46b36afb4d674d2ff23bc070cd70fd42-config\") pod \"kube-apiserver-proxy-ip-10-0-130-101.ec2.internal\" (UID: \"46b36afb4d674d2ff23bc070cd70fd42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.448126 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.448105    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-101.ec2.internal\" not found" node="ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.452225 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.452210    2556 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-101.ec2.internal\" not found" node="ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.520029 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.520011    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.536962 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.536941    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.537027 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.536967    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.537027 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.536983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46b36afb4d674d2ff23bc070cd70fd42-config\") pod \"kube-apiserver-proxy-ip-10-0-130-101.ec2.internal\" (UID: \"46b36afb4d674d2ff23bc070cd70fd42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.537027 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.537012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/46b36afb4d674d2ff23bc070cd70fd42-config\") pod \"kube-apiserver-proxy-ip-10-0-130-101.ec2.internal\" (UID: \"46b36afb4d674d2ff23bc070cd70fd42\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.537133 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.537023    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.537133 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.537040    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20837c157b1fc6e5c1e6bf77fd76888b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal\" (UID: \"20837c157b1fc6e5c1e6bf77fd76888b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.620717 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.620672    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.721209 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.721191    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.751709 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.751693    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.756409 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:42.755936    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:42.822206 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.822177    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:42.922713 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:42.922674    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.023214 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.023191    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.081684 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.081668    2556 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 14 17:54:43.082080 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.081786    2556 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 14 17:54:43.124113 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.124076    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.208151 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.208093    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:49:42 +0000 UTC" deadline="2028-02-15 22:06:19.674598417 +0000 UTC"
Jul 14 17:54:43.208151 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.208122    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13948h11m36.466479589s"
Jul 14 17:54:43.210244 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.210230    2556 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 14 17:54:43.210931 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.210916    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientMemory"
Jul 14 17:54:43.210995 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.210948    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasNoDiskPressure"
Jul 14 17:54:43.210995 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.210958    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeHasSufficientPID"
Jul 14 17:54:43.225040 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.225006    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.325762 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.325743    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.426676 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.426649    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.463743 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.463691    2556 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:54:43.480666 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:43.480634    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b36afb4d674d2ff23bc070cd70fd42.slice/crio-31faf0fe4b9fb3cdbc0479b533f289b24d49cf4139ba6dc5c0600f4a40d5668f WatchSource:0}: Error finding container 31faf0fe4b9fb3cdbc0479b533f289b24d49cf4139ba6dc5c0600f4a40d5668f: Status 404 returned error can't find the container with id 31faf0fe4b9fb3cdbc0479b533f289b24d49cf4139ba6dc5c0600f4a40d5668f
Jul 14 17:54:43.484726 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.484712    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:54:43.526964 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.526930    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.627764 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:43.627740    2556 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-101.ec2.internal\" not found"
Jul 14 17:54:43.638745 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.638726    2556 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:54:43.692533 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.692514    2556 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:54:43.734209 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.734151    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:43.747727 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.746231    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 17:54:43.747727 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.747218    2556 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal"
Jul 14 17:54:43.754463 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:43.754442    2556 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 14 17:54:44.117330 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.117254    2556 apiserver.go:52] "Watching apiserver"
Jul 14 17:54:44.124484 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.124458    2556 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 14 17:54:44.125447 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.125426    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-f4kdn","openshift-dns/node-resolver-2rmp8","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal","openshift-network-diagnostics/network-check-target-cdlkg","openshift-network-operator/iptables-alerter-s8t94","kube-system/konnectivity-agent-p7qc9","openshift-image-registry/node-ca-bs5nf","openshift-multus/multus-9dlzc","openshift-multus/multus-additional-cni-plugins-k9cgm","openshift-multus/network-metrics-daemon-dlnwj","openshift-ovn-kubernetes/ovnkube-node-d4dwf","kube-system/global-pull-secret-syncer-96rbq","kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"]
Jul 14 17:54:44.126865 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.126849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.128517 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.128501    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.129059 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.129041    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.129175 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.129089    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-z9tts\""
Jul 14 17:54:44.129631 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.129595    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 14 17:54:44.129739 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.129651    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.129931 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.129915    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.130365 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130347    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 14 17:54:44.130454 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130438    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.130938 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130814    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 14 17:54:44.130938 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130823    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 14 17:54:44.130938 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130835    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 14 17:54:44.130938 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.130916    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.131181 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.131140    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-tlrs8\""
Jul 14 17:54:44.131252 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.131235    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:44.131358 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.131335    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:44.131916 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.131900    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 14 17:54:44.132003 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.131927    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-n4vwf\""
Jul 14 17:54:44.132003 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.131969    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.132118 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.132026    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.132562 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.132541    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.134058 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134038    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.134310 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134291    2556 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 14 17:54:44.134420 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134363    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.134801 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134778    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xvpwk\""
Jul 14 17:54:44.134801 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 14 17:54:44.134946 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.134930    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.135331 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.135300    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.135855 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.135839    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 14 17:54:44.136854 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.136839    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.137745 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.137727    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 14 17:54:44.137940 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.137923    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-297lb\""
Jul 14 17:54:44.138005 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.137946    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xvdbr\""
Jul 14 17:54:44.138005 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.137930    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 14 17:54:44.138343 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.138324    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 14 17:54:44.138414 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.138369    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.138575 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.138394    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 14 17:54:44.138735 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.138716    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.138791 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.138759    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.139552 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.139531    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h4m9v\""
Jul 14 17:54:44.141602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.139974    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 14 17:54:44.141602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.141032    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.141602 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.141109    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:44.142168 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.142149    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-b6vtx\""
Jul 14 17:54:44.142240 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.142211    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.142293 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.142215    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.142540 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.142519    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.143971 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.143953    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.144042 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.144000    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:44.144235 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.144219    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tf74j\""
Jul 14 17:54:44.144638 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.144623    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 14 17:54:44.144779 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.144763    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 14 17:54:44.145746 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145729    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.145831 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145753    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-agent-certs\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.145831 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145771    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-netns\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.145831 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145791    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-node-log\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.145831 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145821    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-socket-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145841    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-registration-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145878    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-system-cni-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145910    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-systemd-units\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-slash\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145947    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-log-socket\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.145994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145962    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87m74\" (UniqueName: \"kubernetes.io/projected/0888307a-bec3-4560-974f-825103841928-kube-api-access-87m74\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.145999    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lhwq\" (UniqueName: \"kubernetes.io/projected/b5cf6113-b0da-4ffe-85c7-4cd375835967-kube-api-access-6lhwq\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146021    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-etc-selinux\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146052    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0418412-360e-450b-b9e3-89bf9b2632b3-host\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146073    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0418412-360e-450b-b9e3-89bf9b2632b3-serviceca\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146095    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq6bb\" (UniqueName: \"kubernetes.io/projected/3e5d7f17-9f56-405b-a9fb-3140a1504546-kube-api-access-cq6bb\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-device-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-systemd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146189    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146212    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-bin\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-netd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146257    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-sys-fs\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146281    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5cf6113-b0da-4ffe-85c7-4cd375835967-host-slash\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146329    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-kubelet\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-config\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146375    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-script-lib\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146426    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-cnibin\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146446    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-os-release\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146462    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-etc-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146483    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-ovn\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146497    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovn-node-metrics-cert\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146534    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146570    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7hfd\" (UniqueName: \"kubernetes.io/projected/71f4df18-80c9-42b9-8b78-aec1236b5e51-kube-api-access-h7hfd\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146626    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-konnectivity-ca\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.146675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146671    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5cf6113-b0da-4ffe-85c7-4cd375835967-iptables-alerter-script\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146704    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146727    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146773    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6lhr\" (UniqueName: \"kubernetes.io/projected/b0418412-360e-450b-b9e3-89bf9b2632b3-kube-api-access-m6lhr\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146796    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-var-lib-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146842    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.147124 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.146865    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-env-overrides\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.153975 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.153946    2556 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 14 17:54:44.175207 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.175184    2556 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mhrfd"
Jul 14 17:54:44.184260 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.184240    2556 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mhrfd"
Jul 14 17:54:44.235298 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.235276    2556 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 14 17:54:44.247558 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247532    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-netns\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.247668 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247574    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq6bb\" (UniqueName: \"kubernetes.io/projected/3e5d7f17-9f56-405b-a9fb-3140a1504546-kube-api-access-cq6bb\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.247668 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-device-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.247668 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247643    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-sys-fs\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.247668 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247666    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-kubelet\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247689    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247711    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-host\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-script-lib\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247756    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-cnibin\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-os-release\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247803    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-konnectivity-ca\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247826    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-k8s-cni-cncf-io\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247870    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-system-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.247894 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247892    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-device-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247902    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-etc-kubernetes\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-conf\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247956    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-tmp\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.247981    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5cf6113-b0da-4ffe-85c7-4cd375835967-iptables-alerter-script\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248009    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-os-release\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248055    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-modprobe-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248080    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-dbus\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248107    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-env-overrides\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248155    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-cnibin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248178    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248204    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-netns\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.248228 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248227    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-socket-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248252    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-registration-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248279    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-system-cni-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248302    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87m74\" (UniqueName: \"kubernetes.io/projected/0888307a-bec3-4560-974f-825103841928-kube-api-access-87m74\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248327    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-multus-certs\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248350    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-systemd\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248374    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0418412-360e-450b-b9e3-89bf9b2632b3-host\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-bin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248420    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-run\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248444    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-tuned\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248475    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-script-lib\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248482    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3965cd09-acda-428c-bad2-597c0a5b67cf-hosts-file\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248536    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3965cd09-acda-428c-bad2-597c0a5b67cf-tmp-dir\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248563    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-cnibin\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248565    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248600    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-systemd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.248686 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.248688    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-os-release\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.249139 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249099    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-binary-copy\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.249200 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249184    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-sys-fs\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.249246 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249187    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-netns\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.249285 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249246    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-system-cni-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.249327 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249293    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-socket-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.249370 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249337    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-registration-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.249507 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249488    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b0418412-360e-450b-b9e3-89bf9b2632b3-host\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.249585 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249570    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-systemd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249704    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249758    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249752    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-bin\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249824    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-netd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5cf6113-b0da-4ffe-85c7-4cd375835967-host-slash\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249866    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-netd\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249875    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-conf-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249887    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b5cf6113-b0da-4ffe-85c7-4cd375835967-host-slash\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-konnectivity-ca\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249901    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnmpw\" (UniqueName: \"kubernetes.io/projected/06728e84-7be0-447c-9c87-a6a6c70f2837-kube-api-access-vnmpw\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249890    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b5cf6113-b0da-4ffe-85c7-4cd375835967-iptables-alerter-script\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249864    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/71f4df18-80c9-42b9-8b78-aec1236b5e51-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249933    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-kubelet\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249827    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-cni-bin\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.249959    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-config\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250001    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-kubelet\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250028    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250061    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-env-overrides\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250120    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7hfd\" (UniqueName: \"kubernetes.io/projected/71f4df18-80c9-42b9-8b78-aec1236b5e51-kube-api-access-h7hfd\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250166    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-cni-binary-copy\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250313    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-var-lib-kubelet\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250350    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-etc-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250391    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-ovn\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250419    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovn-node-metrics-cert\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovnkube-config\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250444    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250452    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-etc-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250485    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-socket-dir-parent\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-run-ovn\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250564    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-kubernetes\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250591    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48l57\" (UniqueName: \"kubernetes.io/projected/3965cd09-acda-428c-bad2-597c0a5b67cf-kube-api-access-48l57\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.250863 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250672    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6lhr\" (UniqueName: \"kubernetes.io/projected/b0418412-360e-450b-b9e3-89bf9b2632b3-kube-api-access-m6lhr\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250695    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9lpb\" (UniqueName: \"kubernetes.io/projected/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-kube-api-access-z9lpb\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250733    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-var-lib-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250759    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250783    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250785    2556 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250802    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250807    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-agent-certs\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-multus\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250845    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-var-lib-openvswitch\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250855    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysconfig\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250880    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-node-log\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250902    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-hostroot\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250929    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-multus-daemon-config\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2jv8\" (UniqueName: \"kubernetes.io/projected/056792f7-6536-43db-b4f5-b3e6821c5056-kube-api-access-k2jv8\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.250977    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-kubelet-config\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.251736 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251001    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-systemd-units\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251012    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-node-log\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251025    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-slash\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251049    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-log-socket\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251073    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lhwq\" (UniqueName: \"kubernetes.io/projected/b5cf6113-b0da-4ffe-85c7-4cd375835967-kube-api-access-6lhwq\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251121    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-etc-selinux\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251144    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0418412-360e-450b-b9e3-89bf9b2632b3-serviceca\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251166    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-sys\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251190    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-lib-modules\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251244    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-run-ovn-kubernetes\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251321    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-systemd-units\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251367    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-log-socket\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251416    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0888307a-bec3-4560-974f-825103841928-etc-selinux\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251676    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/71f4df18-80c9-42b9-8b78-aec1236b5e51-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.251713    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/3e5d7f17-9f56-405b-a9fb-3140a1504546-host-slash\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.252508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.252511    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b0418412-360e-450b-b9e3-89bf9b2632b3-serviceca\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.254707 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.254687    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/3e5d7f17-9f56-405b-a9fb-3140a1504546-ovn-node-metrics-cert\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.256382 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.256364    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e94b10d6-16cc-457b-ad9b-a5fe64b722dc-agent-certs\") pod \"konnectivity-agent-p7qc9\" (UID: \"e94b10d6-16cc-457b-ad9b-a5fe64b722dc\") " pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.262833 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.262808    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:44.262924 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.262837    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:44.262924 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.262850    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:44.262924 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.262905    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:44.762886137 +0000 UTC m=+2.964441276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:44.264086 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.264063    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq6bb\" (UniqueName: \"kubernetes.io/projected/3e5d7f17-9f56-405b-a9fb-3140a1504546-kube-api-access-cq6bb\") pod \"ovnkube-node-d4dwf\" (UID: \"3e5d7f17-9f56-405b-a9fb-3140a1504546\") " pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.265424 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.265401    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7hfd\" (UniqueName: \"kubernetes.io/projected/71f4df18-80c9-42b9-8b78-aec1236b5e51-kube-api-access-h7hfd\") pod \"multus-additional-cni-plugins-k9cgm\" (UID: \"71f4df18-80c9-42b9-8b78-aec1236b5e51\") " pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.265928 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.265907    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87m74\" (UniqueName: \"kubernetes.io/projected/0888307a-bec3-4560-974f-825103841928-kube-api-access-87m74\") pod \"aws-ebs-csi-driver-node-wj96c\" (UID: \"0888307a-bec3-4560-974f-825103841928\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.267132 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.267106    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lhwq\" (UniqueName: \"kubernetes.io/projected/b5cf6113-b0da-4ffe-85c7-4cd375835967-kube-api-access-6lhwq\") pod \"iptables-alerter-s8t94\" (UID: \"b5cf6113-b0da-4ffe-85c7-4cd375835967\") " pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.267960 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.267939    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6lhr\" (UniqueName: \"kubernetes.io/projected/b0418412-360e-450b-b9e3-89bf9b2632b3-kube-api-access-m6lhr\") pod \"node-ca-bs5nf\" (UID: \"b0418412-360e-450b-b9e3-89bf9b2632b3\") " pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.317616 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.317559    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal" event={"ID":"46b36afb4d674d2ff23bc070cd70fd42","Type":"ContainerStarted","Data":"31faf0fe4b9fb3cdbc0479b533f289b24d49cf4139ba6dc5c0600f4a40d5668f"}
Jul 14 17:54:44.351540 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351517    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-multus\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351658 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351552    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysconfig\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.351658 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351575    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-hostroot\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351658 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351594    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-multus\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351658 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351615    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-multus-daemon-config\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351658 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351638    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2jv8\" (UniqueName: \"kubernetes.io/projected/056792f7-6536-43db-b4f5-b3e6821c5056-kube-api-access-k2jv8\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351662    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-kubelet-config\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351665    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysconfig\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351685    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351710    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-sys\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351731    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-lib-modules\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351753    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-kubelet-config\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351766    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-netns\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351805    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-kubelet\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351812    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-netns\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351842    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351847    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-kubelet\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351640    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-hostroot\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351865    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-host\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.351899 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351892    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-k8s-cni-cncf-io\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351910    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351912    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351950    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-host\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351965    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-lib-modules\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351969    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-system-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.351994    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-system-cni-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-etc-kubernetes\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352024    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352032    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-conf\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.352052    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352069    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-k8s-cni-cncf-io\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352084    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-multus-daemon-config\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.352100    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:44.852086075 +0000 UTC m=+3.053641193 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352115    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-tmp\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352134    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-sys\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352141    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-os-release\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352163    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-modprobe-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.352570 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352174    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-etc-kubernetes\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352143    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-sysctl-conf\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352185    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-dbus\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352214    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-cnibin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352235    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352254    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-modprobe-d\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352262    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-multus-certs\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352285    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-systemd\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352298    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-cnibin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352307    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-bin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352324    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/581bd84f-f8fa-4b81-9cdd-067354af5b90-dbus\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352334    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-run\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352351    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-os-release\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-tuned\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352365    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-run-multus-certs\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352379    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-run\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352388    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3965cd09-acda-428c-bad2-597c0a5b67cf-hosts-file\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352397    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-systemd\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352410    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3965cd09-acda-428c-bad2-597c0a5b67cf-tmp-dir\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352416    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-host-var-lib-cni-bin\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352439    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-conf-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.352451    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352463    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnmpw\" (UniqueName: \"kubernetes.io/projected/06728e84-7be0-447c-9c87-a6a6c70f2837-kube-api-access-vnmpw\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352490    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-cni-binary-copy\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.352506    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:44.852489922 +0000 UTC m=+3.054045041 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352535    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-var-lib-kubelet\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3965cd09-acda-428c-bad2-597c0a5b67cf-hosts-file\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352562    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-socket-dir-parent\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352605    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-kubernetes\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352630    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48l57\" (UniqueName: \"kubernetes.io/projected/3965cd09-acda-428c-bad2-597c0a5b67cf-kube-api-access-48l57\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352659    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9lpb\" (UniqueName: \"kubernetes.io/projected/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-kube-api-access-z9lpb\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352738    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-var-lib-kubelet\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352825    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3965cd09-acda-428c-bad2-597c0a5b67cf-tmp-dir\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352830    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-conf-dir\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-kubernetes\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.353862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.352999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/056792f7-6536-43db-b4f5-b3e6821c5056-cni-binary-copy\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.354607 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.353029    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/056792f7-6536-43db-b4f5-b3e6821c5056-multus-socket-dir-parent\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.354607 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.354200    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-tmp\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.354607 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.354392    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/06728e84-7be0-447c-9c87-a6a6c70f2837-etc-tuned\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.361298 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.361267    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9lpb\" (UniqueName: \"kubernetes.io/projected/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-kube-api-access-z9lpb\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.361595 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.361556    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48l57\" (UniqueName: \"kubernetes.io/projected/3965cd09-acda-428c-bad2-597c0a5b67cf-kube-api-access-48l57\") pod \"node-resolver-2rmp8\" (UID: \"3965cd09-acda-428c-bad2-597c0a5b67cf\") " pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.361671 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.361628    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2jv8\" (UniqueName: \"kubernetes.io/projected/056792f7-6536-43db-b4f5-b3e6821c5056-kube-api-access-k2jv8\") pod \"multus-9dlzc\" (UID: \"056792f7-6536-43db-b4f5-b3e6821c5056\") " pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.361911 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.361892    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnmpw\" (UniqueName: \"kubernetes.io/projected/06728e84-7be0-447c-9c87-a6a6c70f2837-kube-api-access-vnmpw\") pod \"tuned-f4kdn\" (UID: \"06728e84-7be0-447c-9c87-a6a6c70f2837\") " pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.436983 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.436914    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bs5nf"
Jul 14 17:54:44.445764 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.445735    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:54:44.453755 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.453730    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c"
Jul 14 17:54:44.458430 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.458411    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s8t94"
Jul 14 17:54:44.466056 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.466038    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:54:44.472738 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.472718    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k9cgm"
Jul 14 17:54:44.480365 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.480348    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dlzc"
Jul 14 17:54:44.486989 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.486971    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2rmp8"
Jul 14 17:54:44.493767 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.493747    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f4kdn"
Jul 14 17:54:44.620982 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.620952    2556 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 14 17:54:44.856502 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.856426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:44.856502 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.856473    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:44.856507    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856601    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856620    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856629    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856631    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856632    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856678    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:45.856664561 +0000 UTC m=+4.058219683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856694    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:45.856686035 +0000 UTC m=+4.058241153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:44.856723 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:44.856703    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:45.856697907 +0000 UTC m=+4.058253025 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:45.067944 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.067834    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0888307a_bec3_4560_974f_825103841928.slice/crio-d77ef5f36ef6228cce37bafe18dd7d2c3750a4e0ffd0bc0e1621397497a0cfdd WatchSource:0}: Error finding container d77ef5f36ef6228cce37bafe18dd7d2c3750a4e0ffd0bc0e1621397497a0cfdd: Status 404 returned error can't find the container with id d77ef5f36ef6228cce37bafe18dd7d2c3750a4e0ffd0bc0e1621397497a0cfdd
Jul 14 17:54:45.068723 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.068696    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e5d7f17_9f56_405b_a9fb_3140a1504546.slice/crio-8541865cf43fd9156b6ff854811a79e47a94c5964d4d5e907d330075f0d00a79 WatchSource:0}: Error finding container 8541865cf43fd9156b6ff854811a79e47a94c5964d4d5e907d330075f0d00a79: Status 404 returned error can't find the container with id 8541865cf43fd9156b6ff854811a79e47a94c5964d4d5e907d330075f0d00a79
Jul 14 17:54:45.070410 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.069906    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06728e84_7be0_447c_9c87_a6a6c70f2837.slice/crio-850f7ece8db71c76262bb9a86e99cafd8405b46304bf9baea979c38943dcedcc WatchSource:0}: Error finding container 850f7ece8db71c76262bb9a86e99cafd8405b46304bf9baea979c38943dcedcc: Status 404 returned error can't find the container with id 850f7ece8db71c76262bb9a86e99cafd8405b46304bf9baea979c38943dcedcc
Jul 14 17:54:45.074048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.073023    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0418412_360e_450b_b9e3_89bf9b2632b3.slice/crio-4ce7d5699820e68ca485bc9a7f341cae2eb30142e47ef13a5965e218ac56c550 WatchSource:0}: Error finding container 4ce7d5699820e68ca485bc9a7f341cae2eb30142e47ef13a5965e218ac56c550: Status 404 returned error can't find the container with id 4ce7d5699820e68ca485bc9a7f341cae2eb30142e47ef13a5965e218ac56c550
Jul 14 17:54:45.074048 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.073619    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056792f7_6536_43db_b4f5_b3e6821c5056.slice/crio-b2f5634cbf39a831d66ea76aaff9ef8ffb685cdf642f0d9038db173812856701 WatchSource:0}: Error finding container b2f5634cbf39a831d66ea76aaff9ef8ffb685cdf642f0d9038db173812856701: Status 404 returned error can't find the container with id b2f5634cbf39a831d66ea76aaff9ef8ffb685cdf642f0d9038db173812856701
Jul 14 17:54:45.074665 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.074644    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f4df18_80c9_42b9_8b78_aec1236b5e51.slice/crio-25426eb152b79a7f982925f0a580784b6cfe76e206f556ba486f1b566d3a593a WatchSource:0}: Error finding container 25426eb152b79a7f982925f0a580784b6cfe76e206f556ba486f1b566d3a593a: Status 404 returned error can't find the container with id 25426eb152b79a7f982925f0a580784b6cfe76e206f556ba486f1b566d3a593a
Jul 14 17:54:45.075710 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.075429    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5cf6113_b0da_4ffe_85c7_4cd375835967.slice/crio-e1f5eefacbb193761a134e43a48d0aef3f86da1387c8bc0feebd64db4cd5435f WatchSource:0}: Error finding container e1f5eefacbb193761a134e43a48d0aef3f86da1387c8bc0feebd64db4cd5435f: Status 404 returned error can't find the container with id e1f5eefacbb193761a134e43a48d0aef3f86da1387c8bc0feebd64db4cd5435f
Jul 14 17:54:45.186377 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.186252    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:49:44 +0000 UTC" deadline="2028-03-13 19:55:17.9572459 +0000 UTC"
Jul 14 17:54:45.186377 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.186375    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14594h0m32.770873058s"
Jul 14 17:54:45.313560 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.313539    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:45.313653 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.313639    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:45.320432 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.320399    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dlzc" event={"ID":"056792f7-6536-43db-b4f5-b3e6821c5056","Type":"ContainerStarted","Data":"b2f5634cbf39a831d66ea76aaff9ef8ffb685cdf642f0d9038db173812856701"}
Jul 14 17:54:45.321471 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.321449    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bs5nf" event={"ID":"b0418412-360e-450b-b9e3-89bf9b2632b3","Type":"ContainerStarted","Data":"4ce7d5699820e68ca485bc9a7f341cae2eb30142e47ef13a5965e218ac56c550"}
Jul 14 17:54:45.322537 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.322514    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f4kdn" event={"ID":"06728e84-7be0-447c-9c87-a6a6c70f2837","Type":"ContainerStarted","Data":"850f7ece8db71c76262bb9a86e99cafd8405b46304bf9baea979c38943dcedcc"}
Jul 14 17:54:45.323482 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.323465    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c" event={"ID":"0888307a-bec3-4560-974f-825103841928","Type":"ContainerStarted","Data":"d77ef5f36ef6228cce37bafe18dd7d2c3750a4e0ffd0bc0e1621397497a0cfdd"}
Jul 14 17:54:45.325139 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.325121    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal" event={"ID":"46b36afb4d674d2ff23bc070cd70fd42","Type":"ContainerStarted","Data":"f2f9de27ff785c6adca23b3542130ff76598b2a7b7887a8ba44c5b6e0fb038fd"}
Jul 14 17:54:45.326143 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.326126    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2rmp8" event={"ID":"3965cd09-acda-428c-bad2-597c0a5b67cf","Type":"ContainerStarted","Data":"3a784fa9f11a738e3e6d474bbd25ff489ced8587f925f84dee15ac0e616bbfbb"}
Jul 14 17:54:45.327054 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.327037    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8t94" event={"ID":"b5cf6113-b0da-4ffe-85c7-4cd375835967","Type":"ContainerStarted","Data":"e1f5eefacbb193761a134e43a48d0aef3f86da1387c8bc0feebd64db4cd5435f"}
Jul 14 17:54:45.328104 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.328084    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"8541865cf43fd9156b6ff854811a79e47a94c5964d4d5e907d330075f0d00a79"}
Jul 14 17:54:45.329208 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.329186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7qc9" event={"ID":"e94b10d6-16cc-457b-ad9b-a5fe64b722dc","Type":"ContainerStarted","Data":"4f7f23a89debe6ff2b5fc0c282b4d0bbf4da132eaa42171f8ca1075857b17d68"}
Jul 14 17:54:45.330296 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.330275    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerStarted","Data":"25426eb152b79a7f982925f0a580784b6cfe76e206f556ba486f1b566d3a593a"}
Jul 14 17:54:45.340050 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.340016    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-101.ec2.internal" podStartSLOduration=2.340005988 podStartE2EDuration="2.340005988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:54:45.339644075 +0000 UTC m=+3.541199206" watchObservedRunningTime="2026-07-14 17:54:45.340005988 +0000 UTC m=+3.541561127"
Jul 14 17:54:45.607489 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:54:45.607434    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20837c157b1fc6e5c1e6bf77fd76888b.slice/crio-5755309cc0a958c8fa96044c97dc76643798ae328ed860d656d2de322668c8b4 WatchSource:0}: Error finding container 5755309cc0a958c8fa96044c97dc76643798ae328ed860d656d2de322668c8b4: Status 404 returned error can't find the container with id 5755309cc0a958c8fa96044c97dc76643798ae328ed860d656d2de322668c8b4
Jul 14 17:54:45.870875 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.870782    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:45.870875 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.870845    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:45.871091 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:45.870891    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:45.871091 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871061    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:45.871091 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871081    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:45.871091 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871094    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:45.871278 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871165    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:47.871146508 +0000 UTC m=+6.072701631 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:45.871558 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871538    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:45.871685 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871676    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:45.871795 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871756    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:47.871574248 +0000 UTC m=+6.073129385 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:45.871795 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:45.871786    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:47.871775839 +0000 UTC m=+6.073330957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:46.187361 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:46.187258    2556 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:49:44 +0000 UTC" deadline="2028-02-13 00:41:17.302946586 +0000 UTC"
Jul 14 17:54:46.187361 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:46.187290    2556 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13878h46m31.115661327s"
Jul 14 17:54:46.314368 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:46.313885    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:46.314368 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:46.314013    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:46.314368 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:46.314076    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:46.314368 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:46.314208    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:46.339451 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:46.339056    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal" event={"ID":"20837c157b1fc6e5c1e6bf77fd76888b","Type":"ContainerStarted","Data":"5755309cc0a958c8fa96044c97dc76643798ae328ed860d656d2de322668c8b4"}
Jul 14 17:54:47.314005 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:47.313973    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:47.314451 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.314122    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:47.887848 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:47.887814    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:47.888038 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:47.887882    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:47.888038 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:47.887923    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:47.888153 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888053    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:47.888153 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888110    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.888090991 +0000 UTC m=+10.089646113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:47.888482 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888461    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:47.888593 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888520    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.888504322 +0000 UTC m=+10.090059445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:47.888652 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888619    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:47.888652 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888634    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:47.888652 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888647    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:47.888790 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:47.888678    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:51.888669503 +0000 UTC m=+10.090224621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:48.314353 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:48.314175    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:48.314353 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:48.314301    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:48.314733 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:48.314408    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:48.314733 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:48.314514    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:49.314824 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:49.314328    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:49.314824 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:49.314455    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:50.315195 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:50.314732    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:50.315195 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:50.314856    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:50.315958 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:50.315801    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:50.315958 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:50.315915    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:51.313397 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:51.313355    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:51.313570 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.313507    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:51.920910 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:51.920874    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:51.921354 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:51.920936    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:51.921354 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:51.920980    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:51.921354 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921094    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:51.921354 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921154    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:59.92113483 +0000 UTC m=+18.122689953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:51.921537 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921516    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:51.921630 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921562    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:59.921547921 +0000 UTC m=+18.123103042 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:51.921694 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921659    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:51.921694 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921674    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:51.921694 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921685    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:51.921836 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:51.921721    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:54:59.921707997 +0000 UTC m=+18.123263117 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:52.315902 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:52.315829    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:52.316052 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:52.315937    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:52.316202 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:52.316186    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:52.316268 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:52.316255    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:53.314487 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:53.314001    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:53.314487 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:53.314146    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:54.313822 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:54.313795    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:54.313981 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:54.313912    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:54.314265 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:54.314233    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:54.314384 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:54.314340    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:55.314226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:55.314201    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:55.314617 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:55.314300    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:56.314043 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:56.313971    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:56.314189 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:56.314091    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:56.314189 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:56.314178    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:56.314325 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:56.314290    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:57.314131 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:57.314094    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:57.314297 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:57.314221    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:58.314474 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:58.314447    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:58.314474 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:58.314465    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:58.314952 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:58.314565    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:54:58.314952 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:58.314712    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:54:59.313785 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:59.313749    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:59.314011 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.313867    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:54:59.985693 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:59.985665    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:54:59.986119 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:59.985708    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:54:59.986119 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:54:59.985743    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:54:59.986119 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.985836    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:59.986119 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.985887    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:15.985868709 +0000 UTC m=+34.187423829 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:54:59.986318 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986248    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:54:59.986318 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986268    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:54:59.986318 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986280    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:59.986318 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986319    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:15.986308063 +0000 UTC m=+34.187863181 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:54:59.986468 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986372    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:54:59.986468 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:54:59.986403    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:15.986392985 +0000 UTC m=+34.187948106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:55:00.313765 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:00.313697    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:00.313765 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:00.313697    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:00.313966 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:00.313879    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:00.313966 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:00.313884    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:01.314119 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:01.314084    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:01.314640 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:01.314236    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:02.315052 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:02.314571    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:02.315052 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:02.314691    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:02.315052 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:02.314750    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:02.315052 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:02.314836    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:03.313773 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:03.313733    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:03.313948 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:03.313846    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:04.314420 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.314260    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:04.314791 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.314329    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:04.314791 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:04.314501    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:04.314791 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:04.314559    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:04.371616 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.371556    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dlzc" event={"ID":"056792f7-6536-43db-b4f5-b3e6821c5056","Type":"ContainerStarted","Data":"a30a9c8692e5c1e67f6fef0cf31c3f2fdd1b3946183f6e838004fd1e4b31bfc2"}
Jul 14 17:55:04.372941 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.372913    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f4kdn" event={"ID":"06728e84-7be0-447c-9c87-a6a6c70f2837","Type":"ContainerStarted","Data":"a27072204a863ba1455f834a8a4429fa147680b345fefd221c8bef01553f85ce"}
Jul 14 17:55:04.374626 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.374607    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:55:04.374925 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.374907    2556 generic.go:356] "Generic (PLEG): container finished" podID="3e5d7f17-9f56-405b-a9fb-3140a1504546" containerID="7f4a8d91e28f3503335d48e451a56ca3f77146709f251a226403a91140e535ab" exitCode=1
Jul 14 17:55:04.374977 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.374933    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"5fbcab038aa88db149ad30d7ac6bab02fe3d97e8c734078b4bfafacf5379a10f"}
Jul 14 17:55:04.375016 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.374983    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerDied","Data":"7f4a8d91e28f3503335d48e451a56ca3f77146709f251a226403a91140e535ab"}
Jul 14 17:55:04.375016 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.375014    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"eb8022ea38c66a80fdfd56334beb3ab3e1eea2386fe5ab569b4755b96481b71a"}
Jul 14 17:55:04.392218 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.392152    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9dlzc" podStartSLOduration=3.543573087 podStartE2EDuration="22.392114849s" totalImagesPullingTime="18.848541762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:04.391189729 +0000 UTC m=+22.592744868" watchObservedRunningTime="2026-07-14 17:55:04.392114849 +0000 UTC m=+22.593669988"
Jul 14 17:55:04.408955 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:04.408922    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f4kdn" podStartSLOduration=3.585739596 podStartE2EDuration="22.408907763s" totalImagesPullingTime="18.823168167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:04.408552639 +0000 UTC m=+22.610107779" watchObservedRunningTime="2026-07-14 17:55:04.408907763 +0000 UTC m=+22.610462902"
Jul 14 17:55:05.313997 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.313724    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:05.314167 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:05.314059    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:05.378138 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.378103    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2rmp8" event={"ID":"3965cd09-acda-428c-bad2-597c0a5b67cf","Type":"ContainerStarted","Data":"58c858920fd11311422348c15df10a24b5cbd9b36d27971ddb1b9918299756df"}
Jul 14 17:55:05.379518 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.379489    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s8t94" event={"ID":"b5cf6113-b0da-4ffe-85c7-4cd375835967","Type":"ContainerStarted","Data":"4091ee198be6d6de61f6365891b4aa1bb853a7f8240df92946510927061a052c"}
Jul 14 17:55:05.382034 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.382013    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:55:05.382399 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.382376    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"e8f67e5d90cef4f0b9c52a1bc9f712d3ce50963ea9725fbb32428a83f9d3c825"}
Jul 14 17:55:05.382468 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.382442    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"a20179bae1e48c125238e581f9493817c8497b5a226669a569a2cef382664157"}
Jul 14 17:55:05.382513 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.382478    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"17c9b27792632d2fb43dd9ddb6f95f21193e833234913456d99e332fa6745e22"}
Jul 14 17:55:05.383608 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.383573    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-p7qc9" event={"ID":"e94b10d6-16cc-457b-ad9b-a5fe64b722dc","Type":"ContainerStarted","Data":"1d85bb0ade2eb309700c15dc70538211c902d7b1d83f59bee31279325a5643e4"}
Jul 14 17:55:05.384938 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.384916    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="5da7c9d6e0df93c9b228bd7b02dedd36c360c7a47bf39652135a0321bf5cbe4a" exitCode=0
Jul 14 17:55:05.385026 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.384997    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"5da7c9d6e0df93c9b228bd7b02dedd36c360c7a47bf39652135a0321bf5cbe4a"}
Jul 14 17:55:05.386360 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.386331    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bs5nf" event={"ID":"b0418412-360e-450b-b9e3-89bf9b2632b3","Type":"ContainerStarted","Data":"6d60ce6790834ebb8543faf69eec3634a6d6cc322bd583df975a8ddd9cf3fea9"}
Jul 14 17:55:05.391892 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.391864    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c" event={"ID":"0888307a-bec3-4560-974f-825103841928","Type":"ContainerStarted","Data":"2c49a1fe93e9ce40ffe347f64e496c2154c3d8bb31a2e55c1d3b784d4b7ef508"}
Jul 14 17:55:05.393277 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.393256    2556 generic.go:356] "Generic (PLEG): container finished" podID="20837c157b1fc6e5c1e6bf77fd76888b" containerID="2506e570cef54d979f913c831c20becf79fe3f97349218d840c11131b94be6c0" exitCode=0
Jul 14 17:55:05.393362 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.393318    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal" event={"ID":"20837c157b1fc6e5c1e6bf77fd76888b","Type":"ContainerDied","Data":"2506e570cef54d979f913c831c20becf79fe3f97349218d840c11131b94be6c0"}
Jul 14 17:55:05.412175 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.412134    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2rmp8" podStartSLOduration=4.620299885 podStartE2EDuration="23.412122663s" totalImagesPullingTime="18.791822778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:05.395810816 +0000 UTC m=+23.597365955" watchObservedRunningTime="2026-07-14 17:55:05.412122663 +0000 UTC m=+23.613677802"
Jul 14 17:55:05.412305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.412283    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s8t94" podStartSLOduration=4.614033277 podStartE2EDuration="23.412272662s" totalImagesPullingTime="18.798239385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:05.412227161 +0000 UTC m=+23.613782307" watchObservedRunningTime="2026-07-14 17:55:05.412272662 +0000 UTC m=+23.613827801"
Jul 14 17:55:05.467541 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.467507    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-p7qc9" podStartSLOduration=4.674867695 podStartE2EDuration="23.467496556s" totalImagesPullingTime="18.792628861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:05.445930483 +0000 UTC m=+23.647485624" watchObservedRunningTime="2026-07-14 17:55:05.467496556 +0000 UTC m=+23.669051737"
Jul 14 17:55:05.641764 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:05.641737    2556 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 14 17:55:06.209354 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.208789    2556 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:55:05.641758695Z","UUID":"800c24db-aefe-4076-854f-c4c75782da32","Handler":null,"Name":"","Endpoint":""}
Jul 14 17:55:06.210807 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.210779    2556 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 14 17:55:06.210807 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.210810    2556 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 14 17:55:06.314325 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.314283    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:06.314489 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:06.314387    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:06.314816 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.314800    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:06.314909 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:06.314872    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:06.397796 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.397759    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c" event={"ID":"0888307a-bec3-4560-974f-825103841928","Type":"ContainerStarted","Data":"e3e9b0197a33ec265ad809a44b370a571111063453633be8bc056496092f6dd8"}
Jul 14 17:55:06.401176 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.401146    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal" event={"ID":"20837c157b1fc6e5c1e6bf77fd76888b","Type":"ContainerStarted","Data":"08f386b03c1e8baf4254641fb7b974b2627c6689389b2cdc2d81ef835fd052ff"}
Jul 14 17:55:06.422679 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.421649    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-101.ec2.internal" podStartSLOduration=23.421631099 podStartE2EDuration="23.421631099s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:06.420936213 +0000 UTC m=+24.622491354" watchObservedRunningTime="2026-07-14 17:55:06.421631099 +0000 UTC m=+24.623186240"
Jul 14 17:55:06.422679 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:06.421915    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bs5nf" podStartSLOduration=10.056466994 podStartE2EDuration="24.421905917s" totalImagesPullingTime="14.365438923s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:05.482105582 +0000 UTC m=+23.683660704" watchObservedRunningTime="2026-07-14 17:55:06.421905917 +0000 UTC m=+24.623461055"
Jul 14 17:55:07.313531 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:07.313466    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:07.313751 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:07.313559    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:07.407088 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:07.406919    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:55:07.407553 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:07.407527    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"a7841addd58f0471229efed8c686791497295b5f9fd7ec5bc008a8823fb11eb7"}
Jul 14 17:55:07.409619 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:07.409578    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c" event={"ID":"0888307a-bec3-4560-974f-825103841928","Type":"ContainerStarted","Data":"e5f1c3c0e7b135375dc98b44ca8cdc0ce51831bd56ebfbb91356a817011ad52f"}
Jul 14 17:55:08.314237 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:08.314207    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:08.314413 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:08.314321    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:08.314480 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:08.314415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:08.314532 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:08.314498    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:09.313383 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:09.313318    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:09.313858 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:09.313437    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:09.849857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:09.849829    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:55:09.850607 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:09.850479    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:55:09.867814 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:09.867776    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wj96c" podStartSLOduration=6.88919898 podStartE2EDuration="27.867763255s" totalImagesPullingTime="20.978564275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:07.438894093 +0000 UTC m=+25.640449233" watchObservedRunningTime="2026-07-14 17:55:09.867763255 +0000 UTC m=+28.069318395"
Jul 14 17:55:10.314449 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.314398    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:10.314933 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.314403    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:10.314933 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:10.314490    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:10.314933 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:10.314558    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:10.417947 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.417921    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="9c78f2c74249d72296732777ed9deecd5d3cdaac0af8b6d7a1dcc65c71c59513" exitCode=0
Jul 14 17:55:10.418049 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.418008    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"9c78f2c74249d72296732777ed9deecd5d3cdaac0af8b6d7a1dcc65c71c59513"}
Jul 14 17:55:10.421265 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421247    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:55:10.421627 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421606    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"2af0605c5d7feb1bafc4e8bf80c8a701d5063db8aa178f5adda2704c6da9fbd3"}
Jul 14 17:55:10.421710 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421696    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:55:10.421862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421836    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:10.421945 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421921    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:10.421998 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.421973    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:10.422032 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.422013    2556 scope.go:117] "RemoveContainer" containerID="7f4a8d91e28f3503335d48e451a56ca3f77146709f251a226403a91140e535ab"
Jul 14 17:55:10.422188 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.422168    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-p7qc9"
Jul 14 17:55:10.437220 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.437202    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:10.437383 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:10.437369    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:11.314183 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.314155    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:11.314420 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:11.314263    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:11.427546 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.427521    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:55:11.427941 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.427904    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" event={"ID":"3e5d7f17-9f56-405b-a9fb-3140a1504546","Type":"ContainerStarted","Data":"f10d66f6350d12b36bd877b6865833a39ed83e643231152aca00d3a448d5c157"}
Jul 14 17:55:11.466790 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.466742    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf" podStartSLOduration=10.602720487 podStartE2EDuration="29.466727107s" totalImagesPullingTime="18.86400662s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:11.46514479 +0000 UTC m=+29.666699930" watchObservedRunningTime="2026-07-14 17:55:11.466727107 +0000 UTC m=+29.668282247"
Jul 14 17:55:11.810286 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.810253    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96rbq"]
Jul 14 17:55:11.810432 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.810417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:11.810561 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:11.810533    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:11.815425 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.815392    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dlnwj"]
Jul 14 17:55:11.815519 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.815506    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:11.815648 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:11.815628    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:11.816013 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.815994    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdlkg"]
Jul 14 17:55:11.816090 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:11.816061    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:11.816165 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:11.816145    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:12.431836 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:12.431798    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="b863db06a3943adca85621317447ba9ffd1e353181cd59321cf15232239f431f" exitCode=0
Jul 14 17:55:12.432195 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:12.431838    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"b863db06a3943adca85621317447ba9ffd1e353181cd59321cf15232239f431f"}
Jul 14 17:55:13.313571 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:13.313417    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:13.313744 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:13.313431    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:13.313744 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:13.313669    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:13.313744 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:13.313467    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:13.313744 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:13.313735    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:13.313892 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:13.313801    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:14.438239 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:14.438207    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="c5d7928160192f427d6eb6e82bf8b84d83d48b3ca0930f3c406409f4f7cb3c85" exitCode=0
Jul 14 17:55:14.438760 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:14.438255    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"c5d7928160192f427d6eb6e82bf8b84d83d48b3ca0930f3c406409f4f7cb3c85"}
Jul 14 17:55:15.313955 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:15.313870    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:15.313955 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:15.313889    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:15.314187 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:15.313979    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-96rbq" podUID="581bd84f-f8fa-4b81-9cdd-067354af5b90"
Jul 14 17:55:15.314187 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:15.314007    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:15.314187 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:15.314055    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-cdlkg" podUID="2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8"
Jul 14 17:55:15.314187 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:15.314133    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:55:16.014682 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.014643    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.014702    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.014739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014814    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014839    2556 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014838    2556 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014852    2556 projected.go:196] Error preparing data for projected volume kube-api-access-r8q2n for pod openshift-network-diagnostics/network-check-target-cdlkg: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014850    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014898    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret podName:581bd84f-f8fa-4b81-9cdd-067354af5b90 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:48.014882331 +0000 UTC m=+66.216437452 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret") pod "global-pull-secret-syncer-96rbq" (UID: "581bd84f-f8fa-4b81-9cdd-067354af5b90") : object "kube-system"/"original-pull-secret" not registered
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014915    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n podName:2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:48.01490627 +0000 UTC m=+66.216461404 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-r8q2n" (UniqueName: "kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n") pod "network-check-target-cdlkg" (UID: "2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 14 17:55:16.015235 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.014925    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:55:48.014920239 +0000 UTC m=+66.216475357 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 14 17:55:16.619559 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.619482    2556 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-101.ec2.internal" event="NodeReady"
Jul 14 17:55:16.619819 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.619644    2556 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 14 17:55:16.683745 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.683707    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4mgqq"]
Jul 14 17:55:16.686127 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.686105    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.688772 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.688749    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 14 17:55:16.688906 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.688882    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 14 17:55:16.689223 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.689204    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ljwn8\""
Jul 14 17:55:16.698410 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.698389    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-kpkc6"]
Jul 14 17:55:16.700417 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.700399    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4mgqq"]
Jul 14 17:55:16.700519 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.700505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.703272 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.703253    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 14 17:55:16.703807 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.703787    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-pcnzx\""
Jul 14 17:55:16.703956 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.703787    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 14 17:55:16.703956 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.703829    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 14 17:55:16.714866 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.714846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kpkc6"]
Jul 14 17:55:16.821388 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821359    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8cef3db-b23e-4da6-817f-40fbc39fd08f-config-volume\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.821564 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821401    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfj8v\" (UniqueName: \"kubernetes.io/projected/a8cef3db-b23e-4da6-817f-40fbc39fd08f-kube-api-access-hfj8v\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.821564 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821456    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj2tf\" (UniqueName: \"kubernetes.io/projected/d5c2d0a4-f689-468f-856f-280a65f6ec5d-kube-api-access-zj2tf\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.821564 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821507    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8cef3db-b23e-4da6-817f-40fbc39fd08f-tmp-dir\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.821692 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821615    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5c2d0a4-f689-468f-856f-280a65f6ec5d-cert\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.821692 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.821653    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922334 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922254    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfj8v\" (UniqueName: \"kubernetes.io/projected/a8cef3db-b23e-4da6-817f-40fbc39fd08f-kube-api-access-hfj8v\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922334 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922319    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj2tf\" (UniqueName: \"kubernetes.io/projected/d5c2d0a4-f689-468f-856f-280a65f6ec5d-kube-api-access-zj2tf\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.922549 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922336    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8cef3db-b23e-4da6-817f-40fbc39fd08f-tmp-dir\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922549 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922366    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5c2d0a4-f689-468f-856f-280a65f6ec5d-cert\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.922549 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922394    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922549 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922452    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8cef3db-b23e-4da6-817f-40fbc39fd08f-config-volume\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922764 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.922571    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:16.922764 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:16.922692    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:17.422670997 +0000 UTC m=+35.624226130 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:16.922764 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922708    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a8cef3db-b23e-4da6-817f-40fbc39fd08f-tmp-dir\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.922997 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.922980    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a8cef3db-b23e-4da6-817f-40fbc39fd08f-config-volume\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.927279 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.927258    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5c2d0a4-f689-468f-856f-280a65f6ec5d-cert\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:16.938206 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.938087    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfj8v\" (UniqueName: \"kubernetes.io/projected/a8cef3db-b23e-4da6-817f-40fbc39fd08f-kube-api-access-hfj8v\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:16.938206 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:16.938172    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj2tf\" (UniqueName: \"kubernetes.io/projected/d5c2d0a4-f689-468f-856f-280a65f6ec5d-kube-api-access-zj2tf\") pod \"ingress-canary-kpkc6\" (UID: \"d5c2d0a4-f689-468f-856f-280a65f6ec5d\") " pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:17.010197 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.010169    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-kpkc6"
Jul 14 17:55:17.173732 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.173547    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-kpkc6"]
Jul 14 17:55:17.314248 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.314217    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:17.314443 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.314259    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:17.314443 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.314217    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:17.318464 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.318430    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 17:55:17.318789 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.318772    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9r7mf\""
Jul 14 17:55:17.319195 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.319178    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 17:55:17.319279 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.319179    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 17:55:17.319464 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.319447    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 17:55:17.319558 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.319482    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kndq\""
Jul 14 17:55:17.426641 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:17.426537    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:17.426820 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:17.426722    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:17.426820 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:17.426785    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:18.426768549 +0000 UTC m=+36.628323667 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:18.434488 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:18.434448    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:18.434936 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:18.434639    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:18.434936 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:18.434705    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:20.434689989 +0000 UTC m=+38.636245108 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:18.537850 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:55:18.537807    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c2d0a4_f689_468f_856f_280a65f6ec5d.slice/crio-20e57ba04b6f557cb5066122a0da953847cab2c8e6e1219487aa6255f28a2eed WatchSource:0}: Error finding container 20e57ba04b6f557cb5066122a0da953847cab2c8e6e1219487aa6255f28a2eed: Status 404 returned error can't find the container with id 20e57ba04b6f557cb5066122a0da953847cab2c8e6e1219487aa6255f28a2eed
Jul 14 17:55:19.456923 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:19.456886    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="64b40d19132257395c5cf8a44e39f29cf715806f7ab6c11498d003fce412e401" exitCode=0
Jul 14 17:55:19.457371 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:19.456965    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"64b40d19132257395c5cf8a44e39f29cf715806f7ab6c11498d003fce412e401"}
Jul 14 17:55:19.458362 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:19.458334    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kpkc6" event={"ID":"d5c2d0a4-f689-468f-856f-280a65f6ec5d","Type":"ContainerStarted","Data":"20e57ba04b6f557cb5066122a0da953847cab2c8e6e1219487aa6255f28a2eed"}
Jul 14 17:55:20.451324 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:20.451287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:20.451531 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:20.451421    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:20.451531 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:20.451488    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:24.451471169 +0000 UTC m=+42.653026305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:20.464722 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:20.464696    2556 generic.go:356] "Generic (PLEG): container finished" podID="71f4df18-80c9-42b9-8b78-aec1236b5e51" containerID="3ee52da5ec7981ad722b71f6291a0ae3b9c8814818f4bf3245ee87dad44e8cf6" exitCode=0
Jul 14 17:55:20.465106 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:20.464754    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerDied","Data":"3ee52da5ec7981ad722b71f6291a0ae3b9c8814818f4bf3245ee87dad44e8cf6"}
Jul 14 17:55:21.471957 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:21.471730    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" event={"ID":"71f4df18-80c9-42b9-8b78-aec1236b5e51","Type":"ContainerStarted","Data":"e54fee53f7fa9d586daa61e80e3dfcc11637d0071de5e677d77b00dffe897cb0"}
Jul 14 17:55:21.473470 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:21.473443    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-kpkc6" event={"ID":"d5c2d0a4-f689-468f-856f-280a65f6ec5d","Type":"ContainerStarted","Data":"d1d1a01598a3f9a24c9304ebb2aabbfb119c777e840842142598efc628cf8b0c"}
Jul 14 17:55:21.498451 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:21.498408    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k9cgm" podStartSLOduration=10.011591015 podStartE2EDuration="39.49839517s" totalImagesPullingTime="29.486804155s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:21.496772737 +0000 UTC m=+39.698327876" watchObservedRunningTime="2026-07-14 17:55:21.49839517 +0000 UTC m=+39.699950309"
Jul 14 17:55:21.513482 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:21.513447    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-kpkc6" podStartSLOduration=2.861054969 podStartE2EDuration="5.513435671s" totalImagesPullingTime="2.652380702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:21.5129242 +0000 UTC m=+39.714479339" watchObservedRunningTime="2026-07-14 17:55:21.513435671 +0000 UTC m=+39.714990811"
Jul 14 17:55:24.480044 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:24.480012    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:24.480369 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:24.480146    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:24.480369 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:24.480203    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:32.48018614 +0000 UTC m=+50.681741258 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:32.533907 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:32.533874    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:32.534324 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:32.533992    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:32.534324 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:32.534043    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:55:48.534028321 +0000 UTC m=+66.735583444 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:42.443132 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:42.443107    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-d4dwf"
Jul 14 17:55:48.032944 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.032903    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:55:48.033508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.032956    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:48.033508 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.032987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:48.038771 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.038748    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 14 17:55:48.038827 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.038784    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 14 17:55:48.038891 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.038868    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 14 17:55:48.044018 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:48.043994    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 17:55:48.044146 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:48.044073    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:56:52.044047805 +0000 UTC m=+130.245602925 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : secret "metrics-daemon-secret" not found
Jul 14 17:55:48.046285 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.046265    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 14 17:55:48.048689 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.048663    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/581bd84f-f8fa-4b81-9cdd-067354af5b90-original-pull-secret\") pod \"global-pull-secret-syncer-96rbq\" (UID: \"581bd84f-f8fa-4b81-9cdd-067354af5b90\") " pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:48.056950 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.056927    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8q2n\" (UniqueName: \"kubernetes.io/projected/2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8-kube-api-access-r8q2n\") pod \"network-check-target-cdlkg\" (UID: \"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8\") " pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:48.232830 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.232798    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-96rbq"
Jul 14 17:55:48.240572 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.240549    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9r7mf\""
Jul 14 17:55:48.248647 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.248628    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:48.369097 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.369072    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-96rbq"]
Jul 14 17:55:48.372302 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:55:48.372279    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581bd84f_f8fa_4b81_9cdd_067354af5b90.slice/crio-27f87211c525b5801835a5742ba983b73c9a098c72117e77101e0522b0e6de5e WatchSource:0}: Error finding container 27f87211c525b5801835a5742ba983b73c9a098c72117e77101e0522b0e6de5e: Status 404 returned error can't find the container with id 27f87211c525b5801835a5742ba983b73c9a098c72117e77101e0522b0e6de5e
Jul 14 17:55:48.380675 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.380657    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-cdlkg"]
Jul 14 17:55:48.383244 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:55:48.383223    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cefad8e_8ccf_49b0_b0f9_0257f9b9e6f8.slice/crio-8a0049d6f921a3ca169130bdcf924dccb9cc5954140b2eef53fb9ecc8d08cfde WatchSource:0}: Error finding container 8a0049d6f921a3ca169130bdcf924dccb9cc5954140b2eef53fb9ecc8d08cfde: Status 404 returned error can't find the container with id 8a0049d6f921a3ca169130bdcf924dccb9cc5954140b2eef53fb9ecc8d08cfde
Jul 14 17:55:48.531286 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.531261    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdlkg" event={"ID":"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8","Type":"ContainerStarted","Data":"8a0049d6f921a3ca169130bdcf924dccb9cc5954140b2eef53fb9ecc8d08cfde"}
Jul 14 17:55:48.532268 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.532249    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96rbq" event={"ID":"581bd84f-f8fa-4b81-9cdd-067354af5b90","Type":"ContainerStarted","Data":"27f87211c525b5801835a5742ba983b73c9a098c72117e77101e0522b0e6de5e"}
Jul 14 17:55:48.536498 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:48.536482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:55:48.536614 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:48.536602    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:55:48.536661 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:55:48.536651    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:56:20.536635479 +0000 UTC m=+98.738190616 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:55:53.545878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:53.545839    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-cdlkg" event={"ID":"2cefad8e-8ccf-49b0-b0f9-0257f9b9e6f8","Type":"ContainerStarted","Data":"d56c4e2e048e68bd2475b83eac98838f5d6721bd0ac05d079b2233320d6767db"}
Jul 14 17:55:53.546454 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:53.545901    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:55:53.547309 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:53.547285    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-96rbq" event={"ID":"581bd84f-f8fa-4b81-9cdd-067354af5b90","Type":"ContainerStarted","Data":"dc7e562320f935fc3de4a535245d72e81e2f19be4b3e025d350b182749c393aa"}
Jul 14 17:55:53.562897 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:53.562852    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-cdlkg" podStartSLOduration=67.102634363 podStartE2EDuration="1m11.562837782s" totalImagesPullingTime="4.460203419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:53.562209229 +0000 UTC m=+71.763764371" watchObservedRunningTime="2026-07-14 17:55:53.562837782 +0000 UTC m=+71.764392941"
Jul 14 17:55:53.582056 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:55:53.581897    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-96rbq" podStartSLOduration=67.10636445 podStartE2EDuration="1m11.581886827s" totalImagesPullingTime="4.475522377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:55:53.581806112 +0000 UTC m=+71.783361253" watchObservedRunningTime="2026-07-14 17:55:53.581886827 +0000 UTC m=+71.783441965"
Jul 14 17:56:20.553116 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:56:20.552987    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:56:20.553116 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:56:20.553096    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:56:20.553645 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:56:20.553147    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:57:24.553133536 +0000 UTC m=+162.754688654 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:56:24.552142 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:56:24.552111    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-cdlkg"
Jul 14 17:56:52.065795 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:56:52.065748    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:56:52.066272 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:56:52.065886    2556 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 14 17:56:52.066272 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:56:52.065954    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs podName:02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:54.065940511 +0000 UTC m=+252.267495633 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs") pod "network-metrics-daemon-dlnwj" (UID: "02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31") : secret "metrics-daemon-secret" not found
Jul 14 17:57:19.697679 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:57:19.697629    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4mgqq" podUID="a8cef3db-b23e-4da6-817f-40fbc39fd08f"
Jul 14 17:57:19.745016 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:57:19.744981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:57:20.326279 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:57:20.326245    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-dlnwj" podUID="02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31"
Jul 14 17:57:24.577322 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:57:24.577282    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:57:24.577779 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:57:24.577416    2556 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 14 17:57:24.577779 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:57:24.577493    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls podName:a8cef3db-b23e-4da6-817f-40fbc39fd08f nodeName:}" failed. No retries permitted until 2026-07-14 17:59:26.577476153 +0000 UTC m=+284.779031271 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls") pod "dns-default-4mgqq" (UID: "a8cef3db-b23e-4da6-817f-40fbc39fd08f") : secret "dns-default-metrics-tls" not found
Jul 14 17:57:32.314707 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:57:32.314678    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:58:22.248752 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.248660    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"]
Jul 14 17:58:22.250522 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.250505    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.253706 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.253688    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 14 17:58:22.253942 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.253927    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7kx5\""
Jul 14 17:58:22.255994 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.255975    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 14 17:58:22.282226 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.281774    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"]
Jul 14 17:58:22.337216 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.337195    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d84c77e5-cd9c-4a08-acc2-36a378cba51c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.337429 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.337255    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.347716 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.347690    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-s259n"]
Jul 14 17:58:22.349725 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.349707    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6969fbcc8-gndlg"]
Jul 14 17:58:22.349861 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.349844    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.351357 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.351343    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.368482 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368454    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 14 17:58:22.368602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368485    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 14 17:58:22.368602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368495    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 14 17:58:22.368602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368520    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 14 17:58:22.368602 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368538    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 14 17:58:22.368814 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368798    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 14 17:58:22.368862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368805    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 14 17:58:22.368862 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368852    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fz544\""
Jul 14 17:58:22.368958 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368805    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 14 17:58:22.368958 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.368805    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 14 17:58:22.374881 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.374865    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 14 17:58:22.376706 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.376692    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pntrn\""
Jul 14 17:58:22.381360 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.381328    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6969fbcc8-gndlg"]
Jul 14 17:58:22.383045 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.383021    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-s259n"]
Jul 14 17:58:22.383944 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.383926    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 14 17:58:22.438192 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438165    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-stats-auth\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.438305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d84c77e5-cd9c-4a08-acc2-36a378cba51c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.438305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438219    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcjgs\" (UniqueName: \"kubernetes.io/projected/4b333119-42bc-422a-abe2-fdbe41530d5a-kube-api-access-zcjgs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.438305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438234    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-config\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.438305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438248    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhtnh\" (UniqueName: \"kubernetes.io/projected/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-kube-api-access-dhtnh\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.438305 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438295    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-trusted-ca\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438323    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438370    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-default-certificate\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438385    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438399    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-serving-cert\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.438404    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438417    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.438569 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.438465    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:22.938444155 +0000 UTC m=+221.139999289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:22.439014 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.438992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d84c77e5-cd9c-4a08-acc2-36a378cba51c-nginx-conf\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.539055 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.538984    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-default-certificate\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.539055 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539024    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.539055 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539052    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-serving-cert\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.539237 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539070    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.539237 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.539161    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 14 17:58:22.539237 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.539192    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:23.039169862 +0000 UTC m=+221.240725000 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:22.539237 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.539230    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:23.039212362 +0000 UTC m=+221.240767500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : secret "router-metrics-certs-default" not found
Jul 14 17:58:22.539406 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539258    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-stats-auth\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.539406 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539300    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcjgs\" (UniqueName: \"kubernetes.io/projected/4b333119-42bc-422a-abe2-fdbe41530d5a-kube-api-access-zcjgs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.539406 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539316    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-config\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.539406 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539331    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhtnh\" (UniqueName: \"kubernetes.io/projected/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-kube-api-access-dhtnh\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.539406 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.539355    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-trusted-ca\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.540410 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.540372    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-config\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.540808 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.540790    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-trusted-ca\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.541667 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.541648    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-serving-cert\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.541758 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.541727    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-default-certificate\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.542056 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.542036    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-stats-auth\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.549524 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.549500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhtnh\" (UniqueName: \"kubernetes.io/projected/f7e08882-f6ef-4a60-a3a7-cffa01a3452b-kube-api-access-dhtnh\") pod \"console-operator-7866497494-s259n\" (UID: \"f7e08882-f6ef-4a60-a3a7-cffa01a3452b\") " pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.550474 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.550458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcjgs\" (UniqueName: \"kubernetes.io/projected/4b333119-42bc-422a-abe2-fdbe41530d5a-kube-api-access-zcjgs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:22.659299 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.659261    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:22.778755 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.778716    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-s259n"]
Jul 14 17:58:22.781575 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:22.781549    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e08882_f6ef_4a60_a3a7_cffa01a3452b.slice/crio-2ae9725d6914cb7300fde0e11738297d41920839483f5d5f3bf47f1048d3b381 WatchSource:0}: Error finding container 2ae9725d6914cb7300fde0e11738297d41920839483f5d5f3bf47f1048d3b381: Status 404 returned error can't find the container with id 2ae9725d6914cb7300fde0e11738297d41920839483f5d5f3bf47f1048d3b381
Jul 14 17:58:22.899169 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.899140    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-s259n" event={"ID":"f7e08882-f6ef-4a60-a3a7-cffa01a3452b","Type":"ContainerStarted","Data":"2ae9725d6914cb7300fde0e11738297d41920839483f5d5f3bf47f1048d3b381"}
Jul 14 17:58:22.942469 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:22.942441    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:22.942608 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.942571    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:22.942706 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:22.942642    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:23.942628607 +0000 UTC m=+222.144183725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:23.043243 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:23.043218    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:23.043366 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:23.043248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:23.043366 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:23.043354    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 14 17:58:23.043451 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:23.043357    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:24.043343987 +0000 UTC m=+222.244899106 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:23.043451 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:23.043421    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:24.043408181 +0000 UTC m=+222.244963301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : secret "router-metrics-certs-default" not found
Jul 14 17:58:23.952113 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:23.952077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:23.952485 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:23.952226    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:23.952485 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:23.952285    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:25.952269069 +0000 UTC m=+224.153824201 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:24.053505 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:24.053476    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:24.053505 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:24.053512    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:24.053731 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:24.053646    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:26.053630292 +0000 UTC m=+224.255185411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:24.053731 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:24.053651    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 14 17:58:24.053731 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:24.053707    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:26.053688393 +0000 UTC m=+224.255243511 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : secret "router-metrics-certs-default" not found
Jul 14 17:58:25.908034 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:25.908004    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/0.log"
Jul 14 17:58:25.908454 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:25.908046    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7e08882-f6ef-4a60-a3a7-cffa01a3452b" containerID="e60b1d3f057d0dc7e0329836b372ef41d306e6cce892d70fd64a9107651b6a4b" exitCode=255
Jul 14 17:58:25.908454 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:25.908119    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-s259n" event={"ID":"f7e08882-f6ef-4a60-a3a7-cffa01a3452b","Type":"ContainerDied","Data":"e60b1d3f057d0dc7e0329836b372ef41d306e6cce892d70fd64a9107651b6a4b"}
Jul 14 17:58:25.908454 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:25.908362    2556 scope.go:117] "RemoveContainer" containerID="e60b1d3f057d0dc7e0329836b372ef41d306e6cce892d70fd64a9107651b6a4b"
Jul 14 17:58:25.969143 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:25.969111    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:25.969271 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:25.969254    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:25.969319 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:25.969312    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.969292463 +0000 UTC m=+228.170847582 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:26.069907 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.069871    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:26.070036 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.069926    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:26.070036 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:26.070021    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 14 17:58:26.070157 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:26.070051    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:30.070032426 +0000 UTC m=+228.271587545 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:26.070157 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:26.070079    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:30.070065413 +0000 UTC m=+228.271620532 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : secret "router-metrics-certs-default" not found
Jul 14 17:58:26.912349 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.912320    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 17:58:26.912767 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.912715    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/0.log"
Jul 14 17:58:26.912767 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.912752    2556 generic.go:356] "Generic (PLEG): container finished" podID="f7e08882-f6ef-4a60-a3a7-cffa01a3452b" containerID="56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837" exitCode=255
Jul 14 17:58:26.912844 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.912816    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-s259n" event={"ID":"f7e08882-f6ef-4a60-a3a7-cffa01a3452b","Type":"ContainerDied","Data":"56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837"}
Jul 14 17:58:26.912898 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.912884    2556 scope.go:117] "RemoveContainer" containerID="e60b1d3f057d0dc7e0329836b372ef41d306e6cce892d70fd64a9107651b6a4b"
Jul 14 17:58:26.913078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:26.913057    2556 scope.go:117] "RemoveContainer" containerID="56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837"
Jul 14 17:58:26.913249 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:26.913230    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-s259n_openshift-console-operator(f7e08882-f6ef-4a60-a3a7-cffa01a3452b)\"" pod="openshift-console-operator/console-operator-7866497494-s259n" podUID="f7e08882-f6ef-4a60-a3a7-cffa01a3452b"
Jul 14 17:58:27.326405 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.326318    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m77wx"]
Jul 14 17:58:27.329178 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.329162    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.336978 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.336959    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 14 17:58:27.337078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.337048    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 14 17:58:27.337130 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.337078    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 14 17:58:27.337171 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.337134    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5r9n7\""
Jul 14 17:58:27.337171 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.337163    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 14 17:58:27.345529 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.345506    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m77wx"]
Jul 14 17:58:27.479796 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.479750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.479796 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.479806    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.480033 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.479906    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dbe2af4-2154-499d-890c-a605845775b5-crio-socket\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.480033 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.479938    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dbe2af4-2154-499d-890c-a605845775b5-data-volume\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.480033 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.479985    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xsrq\" (UniqueName: \"kubernetes.io/projected/8dbe2af4-2154-499d-890c-a605845775b5-kube-api-access-5xsrq\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580306 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580234    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dbe2af4-2154-499d-890c-a605845775b5-crio-socket\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580306 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580263    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dbe2af4-2154-499d-890c-a605845775b5-data-volume\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580306 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580294    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xsrq\" (UniqueName: \"kubernetes.io/projected/8dbe2af4-2154-499d-890c-a605845775b5-kube-api-access-5xsrq\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580387    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/8dbe2af4-2154-499d-890c-a605845775b5-crio-socket\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580464    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580519    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:27.580596    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:27.580658    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:28.08064337 +0000 UTC m=+226.282198489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : configmap "kube-rbac-proxy" not found
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:27.580673    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.580715    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/8dbe2af4-2154-499d-890c-a605845775b5-data-volume\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.580759 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:27.580740    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:28.080724304 +0000 UTC m=+226.282279425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:27.589283 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.589263    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xsrq\" (UniqueName: \"kubernetes.io/projected/8dbe2af4-2154-499d-890c-a605845775b5-kube-api-access-5xsrq\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:27.916957 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.916930    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 17:58:27.917316 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:27.917246    2556 scope.go:117] "RemoveContainer" containerID="56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837"
Jul 14 17:58:27.917407 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:27.917388    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-s259n_openshift-console-operator(f7e08882-f6ef-4a60-a3a7-cffa01a3452b)\"" pod="openshift-console-operator/console-operator-7866497494-s259n" podUID="f7e08882-f6ef-4a60-a3a7-cffa01a3452b"
Jul 14 17:58:28.084404 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:28.084376    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:28.084558 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:28.084418    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:28.084558 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:28.084490    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:58:28.084558 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:28.084516    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:28.084558 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:28.084541    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.084526261 +0000 UTC m=+227.286081379 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : configmap "kube-rbac-proxy" not found
Jul 14 17:58:28.084558 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:28.084557    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:29.084546353 +0000 UTC m=+227.286101471 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:29.091208 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.091169    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:29.091667 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.091277    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:29.091667 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.091325    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:29.091667 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.091353    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:58:29.091667 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.091394    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:31.091378302 +0000 UTC m=+229.292933421 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:29.091667 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.091410    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:31.091402917 +0000 UTC m=+229.292958037 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : configmap "kube-rbac-proxy" not found
Jul 14 17:58:29.279985 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.279953    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g8bnb"]
Jul 14 17:58:29.281934 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.281916    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.284698 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.284671    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 14 17:58:29.284812 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.284671    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 14 17:58:29.285139 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.285113    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 14 17:58:29.285139 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.285135    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 14 17:58:29.285287 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.285135    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zhwlp\""
Jul 14 17:58:29.292688 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.292660    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g8bnb"]
Jul 14 17:58:29.393441 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.393414    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7wq\" (UniqueName: \"kubernetes.io/projected/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-kube-api-access-2h7wq\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.393548 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.393454    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-key\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.393548 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.393481    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-cabundle\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.493932 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.493896    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7wq\" (UniqueName: \"kubernetes.io/projected/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-kube-api-access-2h7wq\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.493932 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.493934    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-key\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.494085 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.493953    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-cabundle\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.494712 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.494660    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-cabundle\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.496681 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.496658    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-signing-key\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.502857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.502828    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7wq\" (UniqueName: \"kubernetes.io/projected/9bd04f00-0cfe-47a9-b8e6-e6afbc811589-kube-api-access-2h7wq\") pod \"service-ca-79f68b9f4f-g8bnb\" (UID: \"9bd04f00-0cfe-47a9-b8e6-e6afbc811589\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.591551 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.591521    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb"
Jul 14 17:58:29.713026 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.713001    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g8bnb"]
Jul 14 17:58:29.715906 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:29.715875    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd04f00_0cfe_47a9_b8e6_e6afbc811589.slice/crio-9e586c11850c85fe6823650c44e4d03b6155ffd2bb582ba0b683c5f49ec2914e WatchSource:0}: Error finding container 9e586c11850c85fe6823650c44e4d03b6155ffd2bb582ba0b683c5f49ec2914e: Status 404 returned error can't find the container with id 9e586c11850c85fe6823650c44e4d03b6155ffd2bb582ba0b683c5f49ec2914e
Jul 14 17:58:29.914325 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.910264    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2rmp8_3965cd09-acda-428c-bad2-597c0a5b67cf/dns-node-resolver/0.log"
Jul 14 17:58:29.922529 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.922463    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb" event={"ID":"9bd04f00-0cfe-47a9-b8e6-e6afbc811589","Type":"ContainerStarted","Data":"9e586c11850c85fe6823650c44e4d03b6155ffd2bb582ba0b683c5f49ec2914e"}
Jul 14 17:58:29.997138 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:29.997110    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:29.997240 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.997228    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:29.997292 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:29.997278    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:37.997264284 +0000 UTC m=+236.198819403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:30.097643 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:30.097612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:30.097980 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:30.097648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:30.097980 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:30.097756    2556 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 14 17:58:30.097980 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:30.097761    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:38.097748285 +0000 UTC m=+236.299303403 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:30.097980 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:30.097823    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:38.097807826 +0000 UTC m=+236.299362949 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : secret "router-metrics-certs-default" not found
Jul 14 17:58:31.100674 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:31.100640    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bs5nf_b0418412-360e-450b-b9e3-89bf9b2632b3/node-ca/0.log"
Jul 14 17:58:31.106973 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:31.106938    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:31.107113 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:31.107077    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:31.107113 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:31.107095    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:31.107231 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:31.107167    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:35.107147633 +0000 UTC m=+233.308702772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:31.107231 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:31.107168    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:58:31.107231 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:31.107221    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:35.107205237 +0000 UTC m=+233.308760372 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : configmap "kube-rbac-proxy" not found
Jul 14 17:58:31.928393 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:31.928357    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb" event={"ID":"9bd04f00-0cfe-47a9-b8e6-e6afbc811589","Type":"ContainerStarted","Data":"acfa9e2b8efa914533cea98ad750a9b8012ee86a9aeb1d341c48f37807559213"}
Jul 14 17:58:31.945929 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:31.945876    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-g8bnb" podStartSLOduration=1.235492928 podStartE2EDuration="2.945858607s" totalImagesPullingTime="1.710365679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:31.9457528 +0000 UTC m=+230.147307942" watchObservedRunningTime="2026-07-14 17:58:31.945858607 +0000 UTC m=+230.147413751"
Jul 14 17:58:32.101273 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:32.101251    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kpkc6_d5c2d0a4-f689-468f-856f-280a65f6ec5d/serve-healthcheck-canary/0.log"
Jul 14 17:58:32.660336 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:32.660302    2556 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:32.660526 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:32.660357    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:32.660771 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:32.660757    2556 scope.go:117] "RemoveContainer" containerID="56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837"
Jul 14 17:58:32.660945 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:32.660922    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-s259n_openshift-console-operator(f7e08882-f6ef-4a60-a3a7-cffa01a3452b)\"" pod="openshift-console-operator/console-operator-7866497494-s259n" podUID="f7e08882-f6ef-4a60-a3a7-cffa01a3452b"
Jul 14 17:58:35.138965 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:35.138931    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:35.139316 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:35.138991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:35.139316 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:35.139042    2556 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 14 17:58:35.139316 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:35.139106    2556 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:35.139316 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:35.139108    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:43.13909418 +0000 UTC m=+241.340649299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : configmap "kube-rbac-proxy" not found
Jul 14 17:58:35.139316 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:35.139171    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls podName:8dbe2af4-2154-499d-890c-a605845775b5 nodeName:}" failed. No retries permitted until 2026-07-14 17:58:43.139156455 +0000 UTC m=+241.340711580 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m77wx" (UID: "8dbe2af4-2154-499d-890c-a605845775b5") : secret "insights-runtime-extractor-tls" not found
Jul 14 17:58:38.060838 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:38.060794    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:38.061248 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:38.060947    2556 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 14 17:58:38.061248 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:38.061009    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert podName:d84c77e5-cd9c-4a08-acc2-36a378cba51c nodeName:}" failed. No retries permitted until 2026-07-14 17:58:54.060993803 +0000 UTC m=+252.262548924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-jpsxw" (UID: "d84c77e5-cd9c-4a08-acc2-36a378cba51c") : secret "networking-console-plugin-cert" not found
Jul 14 17:58:38.161284 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:38.161244    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:38.161284 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:38.161287    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:38.161441 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:58:38.161423    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle podName:4b333119-42bc-422a-abe2-fdbe41530d5a nodeName:}" failed. No retries permitted until 2026-07-14 17:58:54.161403636 +0000 UTC m=+252.362958756 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle") pod "router-default-6969fbcc8-gndlg" (UID: "4b333119-42bc-422a-abe2-fdbe41530d5a") : configmap references non-existent config key: service-ca.crt
Jul 14 17:58:38.164103 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:38.164077    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b333119-42bc-422a-abe2-fdbe41530d5a-metrics-certs\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:43.198533 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.198489    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:43.199157 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.198548    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:43.199240 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.199194    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/8dbe2af4-2154-499d-890c-a605845775b5-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:43.201311 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.201283    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/8dbe2af4-2154-499d-890c-a605845775b5-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m77wx\" (UID: \"8dbe2af4-2154-499d-890c-a605845775b5\") " pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:43.239925 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.239898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-5r9n7\""
Jul 14 17:58:43.248031 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.248006    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m77wx"
Jul 14 17:58:43.372676 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.372651    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m77wx"]
Jul 14 17:58:43.375479 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:43.375451    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dbe2af4_2154_499d_890c_a605845775b5.slice/crio-415bc2f1c30783af98e456c29e0782754f74323b92a57a1bba3b4590840a892e WatchSource:0}: Error finding container 415bc2f1c30783af98e456c29e0782754f74323b92a57a1bba3b4590840a892e: Status 404 returned error can't find the container with id 415bc2f1c30783af98e456c29e0782754f74323b92a57a1bba3b4590840a892e
Jul 14 17:58:43.970528 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:43.970432    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m77wx" event={"ID":"8dbe2af4-2154-499d-890c-a605845775b5","Type":"ContainerStarted","Data":"415bc2f1c30783af98e456c29e0782754f74323b92a57a1bba3b4590840a892e"}
Jul 14 17:58:45.314115 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:45.314081    2556 scope.go:117] "RemoveContainer" containerID="56e87cc31170ba2c9e0f1e37708d7c34f7f2276c731a83e413e9cb54def9a837"
Jul 14 17:58:46.987724 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:46.987700    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 17:58:46.988106 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:46.987834    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-s259n" event={"ID":"f7e08882-f6ef-4a60-a3a7-cffa01a3452b","Type":"ContainerStarted","Data":"f1e5cea73f7518aca9f63aa8840de6cfb63c94f3626c2e8a0ebd214044cbc36a"}
Jul 14 17:58:46.988209 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:46.988184    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:46.989382 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:46.989359    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m77wx" event={"ID":"8dbe2af4-2154-499d-890c-a605845775b5","Type":"ContainerStarted","Data":"186d8932b16dc4650398c9b7cfe9f48e2a1d96488a91f74678bb56d8b6be892e"}
Jul 14 17:58:47.006437 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:47.006388    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-s259n" podStartSLOduration=22.920148765 podStartE2EDuration="25.006377368s" totalImagesPullingTime="2.086228603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:47.005918698 +0000 UTC m=+245.207473840" watchObservedRunningTime="2026-07-14 17:58:47.006377368 +0000 UTC m=+245.207932508"
Jul 14 17:58:47.672854 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:47.672834    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-s259n"
Jul 14 17:58:47.994625 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:47.994577    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m77wx" event={"ID":"8dbe2af4-2154-499d-890c-a605845775b5","Type":"ContainerStarted","Data":"b3dbf9fc66feba6512755bd2e11637726c5aa9ae3966f7a17d1f5286a8f4f29d"}
Jul 14 17:58:51.006754 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:51.006682    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m77wx" event={"ID":"8dbe2af4-2154-499d-890c-a605845775b5","Type":"ContainerStarted","Data":"32cefb7ad5d1cfd579ef04d013727ab84cd272f345cf385295976fd0eaa22ce6"}
Jul 14 17:58:51.029445 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:51.029348    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m77wx" podStartSLOduration=17.551777067 podStartE2EDuration="24.02932943s" totalImagesPullingTime="6.477552363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:58:27 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:51.028692085 +0000 UTC m=+249.230247225" watchObservedRunningTime="2026-07-14 17:58:51.02932943 +0000 UTC m=+249.230884572"
Jul 14 17:58:54.081565 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.081518    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:54.082053 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.081621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:58:54.084155 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.084130    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31-metrics-certs\") pod \"network-metrics-daemon-dlnwj\" (UID: \"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31\") " pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:58:54.084425 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.084389    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d84c77e5-cd9c-4a08-acc2-36a378cba51c-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-jpsxw\" (UID: \"d84c77e5-cd9c-4a08-acc2-36a378cba51c\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:54.182158 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.182131    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:54.182691 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.182674    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b333119-42bc-422a-abe2-fdbe41530d5a-service-ca-bundle\") pod \"router-default-6969fbcc8-gndlg\" (UID: \"4b333119-42bc-422a-abe2-fdbe41530d5a\") " pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:54.217562 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.217537    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2kndq\""
Jul 14 17:58:54.225498 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.225480    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dlnwj"
Jul 14 17:58:54.350828 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.350805    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dlnwj"]
Jul 14 17:58:54.353480 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:54.353454    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02d1d098_d0d5_4e9d_aeb2_6a7f7b3e4a31.slice/crio-8c6500f014753955935c0869d288b89e8a82dddd215aa22c39a74a8c4d01f10b WatchSource:0}: Error finding container 8c6500f014753955935c0869d288b89e8a82dddd215aa22c39a74a8c4d01f10b: Status 404 returned error can't find the container with id 8c6500f014753955935c0869d288b89e8a82dddd215aa22c39a74a8c4d01f10b
Jul 14 17:58:54.360632 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.360612    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-b7kx5\""
Jul 14 17:58:54.368619 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.368596    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"
Jul 14 17:58:54.466297 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.466275    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-fz544\""
Jul 14 17:58:54.474041 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.474019    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:54.482739 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.482716    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-jpsxw"]
Jul 14 17:58:54.485752 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:54.485730    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd84c77e5_cd9c_4a08_acc2_36a378cba51c.slice/crio-ed3201698bb44763f19772bb228a2801a2466eb2bad348273d6f74e1bcc714b5 WatchSource:0}: Error finding container ed3201698bb44763f19772bb228a2801a2466eb2bad348273d6f74e1bcc714b5: Status 404 returned error can't find the container with id ed3201698bb44763f19772bb228a2801a2466eb2bad348273d6f74e1bcc714b5
Jul 14 17:58:54.600531 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:54.600502    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6969fbcc8-gndlg"]
Jul 14 17:58:54.603307 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:58:54.603285    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b333119_42bc_422a_abe2_fdbe41530d5a.slice/crio-2506cd20d153ef93a69b69e9dbabf65d88663c15c388433c63378b25d218f2ef WatchSource:0}: Error finding container 2506cd20d153ef93a69b69e9dbabf65d88663c15c388433c63378b25d218f2ef: Status 404 returned error can't find the container with id 2506cd20d153ef93a69b69e9dbabf65d88663c15c388433c63378b25d218f2ef
Jul 14 17:58:55.021496 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.021414    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dlnwj" event={"ID":"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31","Type":"ContainerStarted","Data":"8c6500f014753955935c0869d288b89e8a82dddd215aa22c39a74a8c4d01f10b"}
Jul 14 17:58:55.022722 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.022694    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw" event={"ID":"d84c77e5-cd9c-4a08-acc2-36a378cba51c","Type":"ContainerStarted","Data":"ed3201698bb44763f19772bb228a2801a2466eb2bad348273d6f74e1bcc714b5"}
Jul 14 17:58:55.024133 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.024109    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6969fbcc8-gndlg" event={"ID":"4b333119-42bc-422a-abe2-fdbe41530d5a","Type":"ContainerStarted","Data":"e4ae8372673f0dd18285a5796a4f5b00f8593e3a6f5e4ab12619dcd443dedb5e"}
Jul 14 17:58:55.024222 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.024152    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6969fbcc8-gndlg" event={"ID":"4b333119-42bc-422a-abe2-fdbe41530d5a","Type":"ContainerStarted","Data":"2506cd20d153ef93a69b69e9dbabf65d88663c15c388433c63378b25d218f2ef"}
Jul 14 17:58:55.045833 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.045775    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6969fbcc8-gndlg" podStartSLOduration=33.045757463 podStartE2EDuration="33.045757463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:55.045141434 +0000 UTC m=+253.246696575" watchObservedRunningTime="2026-07-14 17:58:55.045757463 +0000 UTC m=+253.247312605"
Jul 14 17:58:55.475280 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.475240    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:55.478214 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:55.478180    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:56.028748 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:56.028715    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw" event={"ID":"d84c77e5-cd9c-4a08-acc2-36a378cba51c","Type":"ContainerStarted","Data":"6ab729642ec810d0c9a5944790628c54cd4da60ed93222e70f7b06d1c24102ab"}
Jul 14 17:58:56.028931 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:56.028892    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:56.030142 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:56.030121    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6969fbcc8-gndlg"
Jul 14 17:58:56.047709 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:58:56.047666    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-jpsxw" podStartSLOduration=32.944177285 podStartE2EDuration="34.04765246s" totalImagesPullingTime="1.103475175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:56.047449724 +0000 UTC m=+254.249004865" watchObservedRunningTime="2026-07-14 17:58:56.04765246 +0000 UTC m=+254.249207604"
Jul 14 17:59:00.044252 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:00.044209    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dlnwj" event={"ID":"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31","Type":"ContainerStarted","Data":"af0ac5edb0fed23b34f12ada5d96e913b046f09cca145ed33b1ca8edfd6ecd1e"}
Jul 14 17:59:00.044676 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:00.044274    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dlnwj" event={"ID":"02d1d098-d0d5-4e9d-aeb2-6a7f7b3e4a31","Type":"ContainerStarted","Data":"55f48dadc865268fae98a08255ddcce466933ff60e20687780b7625cc37ca94a"}
Jul 14 17:59:00.063925 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:00.063870    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dlnwj" podStartSLOduration=253.314617186 podStartE2EDuration="4m18.063853808s" totalImagesPullingTime="4.749236622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:54:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:00.062914882 +0000 UTC m=+258.264470024" watchObservedRunningTime="2026-07-14 17:59:00.063853808 +0000 UTC m=+258.265408948"
Jul 14 17:59:22.746010 ip-10-0-130-101 kubenswrapper[2556]: E0714 17:59:22.745929    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-4mgqq" podUID="a8cef3db-b23e-4da6-817f-40fbc39fd08f"
Jul 14 17:59:23.121941 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:23.121915    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:26.582883 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:26.582844    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:26.585387 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:26.585362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a8cef3db-b23e-4da6-817f-40fbc39fd08f-metrics-tls\") pod \"dns-default-4mgqq\" (UID: \"a8cef3db-b23e-4da6-817f-40fbc39fd08f\") " pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:26.726857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:26.726829    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ljwn8\""
Jul 14 17:59:26.733287 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:26.733266    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:26.876478 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:26.876451    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4mgqq"]
Jul 14 17:59:26.879672 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:59:26.879638    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cef3db_b23e_4da6_817f_40fbc39fd08f.slice/crio-a9d117c86e06c97464df94453852c0574a0d4476f01e387566efaaa05cde4aec WatchSource:0}: Error finding container a9d117c86e06c97464df94453852c0574a0d4476f01e387566efaaa05cde4aec: Status 404 returned error can't find the container with id a9d117c86e06c97464df94453852c0574a0d4476f01e387566efaaa05cde4aec
Jul 14 17:59:27.136956 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:27.136886    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4mgqq" event={"ID":"a8cef3db-b23e-4da6-817f-40fbc39fd08f","Type":"ContainerStarted","Data":"a9d117c86e06c97464df94453852c0574a0d4476f01e387566efaaa05cde4aec"}
Jul 14 17:59:29.146568 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:29.146535    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4mgqq" event={"ID":"a8cef3db-b23e-4da6-817f-40fbc39fd08f","Type":"ContainerStarted","Data":"5cddb77796c9a517f94d87c9b0ff47c9194c60dfcee414b3ff8c9ecb16b91b53"}
Jul 14 17:59:29.146961 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:29.146604    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4mgqq" event={"ID":"a8cef3db-b23e-4da6-817f-40fbc39fd08f","Type":"ContainerStarted","Data":"55c010f35a2e0c9284676318d2a12c425081126007f68a72e9004220103abf85"}
Jul 14 17:59:29.166423 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:29.166375    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4mgqq" podStartSLOduration=251.42374548 podStartE2EDuration="4m13.16636033s" totalImagesPullingTime="1.74261485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:55:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:29.165548023 +0000 UTC m=+287.367103164" watchObservedRunningTime="2026-07-14 17:59:29.16636033 +0000 UTC m=+287.367915471"
Jul 14 17:59:30.149471 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:30.149432    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:40.154535 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:40.154510    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4mgqq"
Jul 14 17:59:42.144202 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:42.144170    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 17:59:42.144632 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:42.144299    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 17:59:42.150707 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:42.150684    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:59:42.151173 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:42.151143    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 17:59:42.153379 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:42.153360    2556 kubelet.go:1648] "Image garbage collection succeeded"
Jul 14 17:59:54.278915 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.278883    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 17:59:54.280942 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.280925    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.284122 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.284019    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 14 17:59:54.284122 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.284035    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 14 17:59:54.285222 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.285181    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 14 17:59:54.287710 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.285441    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-tqzm2\""
Jul 14 17:59:54.287710 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.285743    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 14 17:59:54.292336 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.288633    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 14 17:59:54.292336 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.289610    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 14 17:59:54.292336 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.289813    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 14 17:59:54.295116 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.295095    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 14 17:59:54.300491 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.300469    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 17:59:54.347035 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347004    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347131 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347041    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347131 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347071    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347215 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347131    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347215 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347156    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs4bp\" (UniqueName: \"kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347215 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347174    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.347215 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.347193    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.390335 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.390310    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b49f4f54f-cq6xc"]
Jul 14 17:59:54.392336 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.392323    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.400349 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.400330    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 14 17:59:54.400802 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.400776    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rr2k7\""
Jul 14 17:59:54.401296 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.401282    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 14 17:59:54.402667 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.402646    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 14 17:59:54.405038 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.405022    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 14 17:59:54.415984 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.415964    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b49f4f54f-cq6xc"]
Jul 14 17:59:54.447773 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447751    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.447857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447775    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs4bp\" (UniqueName: \"kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.447857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447793    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-certificates\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.447857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447810    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.447857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-trusted-ca\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.447857 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447855    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447890    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26w4t\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-kube-api-access-26w4t\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447930    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-tls\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.447969    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-installation-pull-secrets\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448011    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448078 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448035    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8e5fffb-1b22-4802-991b-7036d56a4419-ca-trust-extracted\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448318 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448089    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448318 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448192    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-image-registry-private-configuration\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448318 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448244    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448318 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448275    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-bound-sa-token\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.448689 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448634    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448802 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448748    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448853 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448846    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.448907 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.448863    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.450326 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.450305    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.450411 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.450339    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.469307 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.469279    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs4bp\" (UniqueName: \"kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp\") pod \"console-7cf667b8ff-2t4r6\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") " pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.549024 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.548971    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-certificates\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549024 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.548998    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-trusted-ca\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549024 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549021    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26w4t\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-kube-api-access-26w4t\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549384 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549042    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-tls\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549384 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549069    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-installation-pull-secrets\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549384 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549106    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8e5fffb-1b22-4802-991b-7036d56a4419-ca-trust-extracted\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549384 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549141    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-image-registry-private-configuration\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549384 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549171    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-bound-sa-token\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549661 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8e5fffb-1b22-4802-991b-7036d56a4419-ca-trust-extracted\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549878 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549841    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-certificates\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.549956 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.549899    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8e5fffb-1b22-4802-991b-7036d56a4419-trusted-ca\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.551834 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.551807    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-image-registry-private-configuration\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.552030 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.552009    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-registry-tls\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.552088 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.552035    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8e5fffb-1b22-4802-991b-7036d56a4419-installation-pull-secrets\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.556898 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.556875    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26w4t\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-kube-api-access-26w4t\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.557092 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.557070    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8e5fffb-1b22-4802-991b-7036d56a4419-bound-sa-token\") pod \"image-registry-5b49f4f54f-cq6xc\" (UID: \"f8e5fffb-1b22-4802-991b-7036d56a4419\") " pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.594075 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.594050    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 17:59:54.700918 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.700895    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:54.723870 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.723847    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 17:59:54.726971 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:59:54.726944    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod581d2e50_d9ce_4d5b_98bf_3a34f2d4d7ad.slice/crio-40d9de43663f2e641ad332fbdf36bab65c06d37e1ae9c559b60fd5d1020bec5a WatchSource:0}: Error finding container 40d9de43663f2e641ad332fbdf36bab65c06d37e1ae9c559b60fd5d1020bec5a: Status 404 returned error can't find the container with id 40d9de43663f2e641ad332fbdf36bab65c06d37e1ae9c559b60fd5d1020bec5a
Jul 14 17:59:54.728947 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.728932    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 17:59:54.823654 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:54.823596    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b49f4f54f-cq6xc"]
Jul 14 17:59:54.828168 ip-10-0-130-101 kubenswrapper[2556]: W0714 17:59:54.828143    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e5fffb_1b22_4802_991b_7036d56a4419.slice/crio-f0026ac8394612f2405c8798d9eddee87cfa880d516be86967dc1d8df940845f WatchSource:0}: Error finding container f0026ac8394612f2405c8798d9eddee87cfa880d516be86967dc1d8df940845f: Status 404 returned error can't find the container with id f0026ac8394612f2405c8798d9eddee87cfa880d516be86967dc1d8df940845f
Jul 14 17:59:55.228164 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:55.228108    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf667b8ff-2t4r6" event={"ID":"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad","Type":"ContainerStarted","Data":"40d9de43663f2e641ad332fbdf36bab65c06d37e1ae9c559b60fd5d1020bec5a"}
Jul 14 17:59:55.229881 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:55.229849    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc" event={"ID":"f8e5fffb-1b22-4802-991b-7036d56a4419","Type":"ContainerStarted","Data":"858cc08bd33a9d926e1d5a66e64d775bccf4c07c3f3e0fc3f1d243077c649943"}
Jul 14 17:59:55.230027 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:55.229899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc" event={"ID":"f8e5fffb-1b22-4802-991b-7036d56a4419","Type":"ContainerStarted","Data":"f0026ac8394612f2405c8798d9eddee87cfa880d516be86967dc1d8df940845f"}
Jul 14 17:59:55.230027 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:55.229927    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 17:59:55.255576 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:55.255526    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc" podStartSLOduration=1.255507431 podStartE2EDuration="1.255507431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:59:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:55.254533415 +0000 UTC m=+313.456088570" watchObservedRunningTime="2026-07-14 17:59:55.255507431 +0000 UTC m=+313.457062573"
Jul 14 17:59:58.241311 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:58.241279    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf667b8ff-2t4r6" event={"ID":"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad","Type":"ContainerStarted","Data":"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"}
Jul 14 17:59:58.260557 ip-10-0-130-101 kubenswrapper[2556]: I0714 17:59:58.260507    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cf667b8ff-2t4r6" podStartSLOduration=1.55771562 podStartE2EDuration="4.260491937s" totalImagesPullingTime="2.702776317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:58.259974867 +0000 UTC m=+316.461530007" watchObservedRunningTime="2026-07-14 17:59:58.260491937 +0000 UTC m=+316.462047076"
Jul 14 18:00:00.595917 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.595882    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fg7dw"]
Jul 14 18:00:00.599469 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.599449    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.603482 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.603458    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 14 18:00:00.604465 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604248    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 14 18:00:00.604465 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604284    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7kfx4\""
Jul 14 18:00:00.604465 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604327    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 14 18:00:00.604687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604485    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 14 18:00:00.604687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604559    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 14 18:00:00.604783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.604705    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 14 18:00:00.694797 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694778    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-textfile\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.694909 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694832    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.694909 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694857    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttgxx\" (UniqueName: \"kubernetes.io/projected/b28a5863-4f50-4da9-afc0-e5e3047392f7-kube-api-access-ttgxx\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695005 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694914    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-root\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695005 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694955    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-tls\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695005 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.694980    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-sys\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695112 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.695005    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695112 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.695044    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-metrics-client-ca\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.695112 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.695071    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-wtmp\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795650 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795621    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-sys\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795660    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795694    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-metrics-client-ca\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795718    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-wtmp\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795725    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-sys\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795751    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-textfile\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795797    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.795835 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795822    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttgxx\" (UniqueName: \"kubernetes.io/projected/b28a5863-4f50-4da9-afc0-e5e3047392f7-kube-api-access-ttgxx\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796172 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795839    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-wtmp\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796172 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795848    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-root\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796172 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795908    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b28a5863-4f50-4da9-afc0-e5e3047392f7-root\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796172 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.795938    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-tls\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796172 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.796117    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-textfile\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796374 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.796280    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-metrics-client-ca\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.796469 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.796451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-accelerators-collector-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.798143 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.798118    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-tls\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.798297 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.798282    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b28a5863-4f50-4da9-afc0-e5e3047392f7-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.817638 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.817614    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttgxx\" (UniqueName: \"kubernetes.io/projected/b28a5863-4f50-4da9-afc0-e5e3047392f7-kube-api-access-ttgxx\") pod \"node-exporter-fg7dw\" (UID: \"b28a5863-4f50-4da9-afc0-e5e3047392f7\") " pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.910105 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:00.910083    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fg7dw"
Jul 14 18:00:00.920897 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:00:00.920873    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28a5863_4f50_4da9_afc0_e5e3047392f7.slice/crio-7e3226f1a0724fcc814947a537cb8ac4cbdf5ad946d8600a0f1660749f924aef WatchSource:0}: Error finding container 7e3226f1a0724fcc814947a537cb8ac4cbdf5ad946d8600a0f1660749f924aef: Status 404 returned error can't find the container with id 7e3226f1a0724fcc814947a537cb8ac4cbdf5ad946d8600a0f1660749f924aef
Jul 14 18:00:01.255253 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:01.255184    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fg7dw" event={"ID":"b28a5863-4f50-4da9-afc0-e5e3047392f7","Type":"ContainerStarted","Data":"7e3226f1a0724fcc814947a537cb8ac4cbdf5ad946d8600a0f1660749f924aef"}
Jul 14 18:00:03.263861 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:03.263827    2556 generic.go:356] "Generic (PLEG): container finished" podID="b28a5863-4f50-4da9-afc0-e5e3047392f7" containerID="5d531d617f299b8c3ce2d47c42aee907da15ee39568e019ef33a27022d0e32af" exitCode=0
Jul 14 18:00:03.264311 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:03.263895    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fg7dw" event={"ID":"b28a5863-4f50-4da9-afc0-e5e3047392f7","Type":"ContainerDied","Data":"5d531d617f299b8c3ce2d47c42aee907da15ee39568e019ef33a27022d0e32af"}
Jul 14 18:00:04.268963 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.268928    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fg7dw" event={"ID":"b28a5863-4f50-4da9-afc0-e5e3047392f7","Type":"ContainerStarted","Data":"566bce68390037207b2b30ca05ff4d58949a1c2f17eab706b20d0ca20a9cc45b"}
Jul 14 18:00:04.269330 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.269009    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fg7dw" event={"ID":"b28a5863-4f50-4da9-afc0-e5e3047392f7","Type":"ContainerStarted","Data":"6c3d2eb701b53c42264ac85845f2b7cc5c32ed2762324cabd7285d34a5625824"}
Jul 14 18:00:04.297354 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.297308    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fg7dw" podStartSLOduration=2.757535828 podStartE2EDuration="4.297293953s" totalImagesPullingTime="1.471428551s" totalInitContainerRuntime="68.329574ms" isStatefulPod=true podCreationTimestamp="2026-07-14 18:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:04.296045833 +0000 UTC m=+322.497600965" watchObservedRunningTime="2026-07-14 18:00:04.297293953 +0000 UTC m=+322.498849092"
Jul 14 18:00:04.595059 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.594993    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:04.595059 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.595039    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:04.599423 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:04.599399    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:05.276814 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:05.276788    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:06.728291 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.728263    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 18:00:06.731411 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.731394    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.736624 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736577    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 14 18:00:06.736749 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736639    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 14 18:00:06.736749 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736658    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 14 18:00:06.736749 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736685    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 14 18:00:06.736749 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736639    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 14 18:00:06.736944 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.736898    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 14 18:00:06.737433 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737412    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-d72q5\""
Jul 14 18:00:06.737601 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737563    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 14 18:00:06.737677 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737569    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 14 18:00:06.737727 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737715    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 14 18:00:06.737727 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737723    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 14 18:00:06.737883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737861    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 14 18:00:06.737883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.737872    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 14 18:00:06.738076 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.738059    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 14 18:00:06.738253 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.738238    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-6jkpkuikvsi06\""
Jul 14 18:00:06.739419 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.739404    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 14 18:00:06.742436 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.742417    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 14 18:00:06.751379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.751358    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 18:00:06.837537 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837512    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhj4d\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-kube-api-access-dhj4d\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837544    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837605    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837638    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837662    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837683    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837705    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-web-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837730    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837756    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837800    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config-out\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837839    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.837872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837864    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837887    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837944    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.837972    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.838008    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.838023    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.838036    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.838077    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.838109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.838097    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938691 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938694    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938708    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938724    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhj4d\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-kube-api-access-dhj4d\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938758    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938997 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938911    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938997 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938963    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.938997 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.938991    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939015    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939038    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-web-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939068    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939092    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939124    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config-out\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939162    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939180    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939208    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939245    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939273    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939301    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939841 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939504    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.939841 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939547    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.940085 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.939986    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.942843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942299    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.942843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942309    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.942843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942388    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.942843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942699    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.942843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942771    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.943141 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.942889    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.943646 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.943334    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.943646 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.943435    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config-out\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.943775 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.943723    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.944142 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.944113    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.944286 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.944268    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.944359 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.944282    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.944458 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.944423    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.944636 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.944616    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-web-config\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.945237 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.945220    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.945853 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.945836    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:06.947290 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:06.947271    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhj4d\" (UniqueName: \"kubernetes.io/projected/af77b72e-38e0-4f1b-9fdf-1cf9c4948da3-kube-api-access-dhj4d\") pod \"prometheus-k8s-0\" (UID: \"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:07.045898 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:07.045841    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:07.180804 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:07.180773    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 14 18:00:07.183704 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:00:07.183674    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf77b72e_38e0_4f1b_9fdf_1cf9c4948da3.slice/crio-318a9fca913e64af365961aa5f5e87880ba9efc1082807647dc7b573b4aaaabc WatchSource:0}: Error finding container 318a9fca913e64af365961aa5f5e87880ba9efc1082807647dc7b573b4aaaabc: Status 404 returned error can't find the container with id 318a9fca913e64af365961aa5f5e87880ba9efc1082807647dc7b573b4aaaabc
Jul 14 18:00:07.280602 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:07.280555    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"318a9fca913e64af365961aa5f5e87880ba9efc1082807647dc7b573b4aaaabc"}
Jul 14 18:00:09.289623 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:09.289573    2556 generic.go:356] "Generic (PLEG): container finished" podID="af77b72e-38e0-4f1b-9fdf-1cf9c4948da3" containerID="3579bb386ad9e056dff348bd833d13c53fd69e80bebc17f8807a6947d18a1e80" exitCode=0
Jul 14 18:00:09.289942 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:09.289682    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerDied","Data":"3579bb386ad9e056dff348bd833d13c53fd69e80bebc17f8807a6947d18a1e80"}
Jul 14 18:00:13.308171 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:13.308135    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"0e41a92ee09d6418508a9daef23428140d152be1cfd219ee5e8e2bc06fe97484"}
Jul 14 18:00:13.308510 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:13.308225    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"4fce629f9a14e8fe96cd3213f12c18cdeed64938e548e1459cc47c458f2ba857"}
Jul 14 18:00:16.238305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:16.238273    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b49f4f54f-cq6xc"
Jul 14 18:00:17.325467 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:17.325435    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"356c71bbe8886a4c682b3ed9864666482cafe99e40215b5fb5bde049e4b45d79"}
Jul 14 18:00:17.325823 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:17.325507    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"ad29b74f6f10f4954eef1a84e7ee0efc2542fba0cbf5d2ddace6f9d52cf821d4"}
Jul 14 18:00:17.325823 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:17.325552    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"93c7b1a89bb397e08b1caec03e9f8f52407ef25348169f0011e4b563abf6e197"}
Jul 14 18:00:17.325823 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:17.325610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"af77b72e-38e0-4f1b-9fdf-1cf9c4948da3","Type":"ContainerStarted","Data":"8be429f15516edb48b63078c94f849b8b6a69c254448b1f2562dbf584e0489b8"}
Jul 14 18:00:17.364699 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:17.364656    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.780977066 podStartE2EDuration="11.364642647s" totalImagesPullingTime="8.566302819s" totalInitContainerRuntime="17.362762ms" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:17.363405755 +0000 UTC m=+335.564960895" watchObservedRunningTime="2026-07-14 18:00:17.364642647 +0000 UTC m=+335.566197786"
Jul 14 18:00:22.046139 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:22.046108    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:00:25.704894 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:25.704857    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 18:00:50.726457 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:50.726354    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cf667b8ff-2t4r6" podUID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" containerName="console" containerID="cri-o://244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8" gracePeriod=15
Jul 14 18:00:50.980827 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:50.980774    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:51.059735 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059708    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.059851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059742    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.059851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059765    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vs4bp\" (UniqueName: \"kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.059851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.060002 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059857    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.060002 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.059888    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.060097 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060043    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config\") pod \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\" (UID: \"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad\") "
Jul 14 18:00:51.060255 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060177    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:00:51.060255 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060188    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:00:51.060387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060317    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:00:51.060387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060355    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config" (OuterVolumeSpecName: "console-config") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:00:51.060665 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060546    2556 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.060665 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060565    2556 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-config\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.060665 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060600    2556 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-oauth-serving-cert\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.060665 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.060613    2556 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-trusted-ca-bundle\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.062071 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.062044    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:00:51.062165 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.062089    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:00:51.062203 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.062152    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp" (OuterVolumeSpecName: "kube-api-access-vs4bp") pod "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" (UID: "581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad"). InnerVolumeSpecName "kube-api-access-vs4bp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:00:51.161857 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.161834    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vs4bp\" (UniqueName: \"kubernetes.io/projected/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-kube-api-access-vs4bp\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.161857 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.161855    2556 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-oauth-config\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.161972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.161864    2556 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad-console-serving-cert\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:00:51.436838 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.436810    2556 generic.go:356] "Generic (PLEG): container finished" podID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" containerID="244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8" exitCode=0
Jul 14 18:00:51.436926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.436852    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cf667b8ff-2t4r6"
Jul 14 18:00:51.436926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.436855    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf667b8ff-2t4r6" event={"ID":"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad","Type":"ContainerDied","Data":"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"}
Jul 14 18:00:51.436926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.436891    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cf667b8ff-2t4r6" event={"ID":"581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad","Type":"ContainerDied","Data":"40d9de43663f2e641ad332fbdf36bab65c06d37e1ae9c559b60fd5d1020bec5a"}
Jul 14 18:00:51.436926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.436911    2556 scope.go:117] "RemoveContainer" containerID="244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"
Jul 14 18:00:51.445367 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.445352    2556 scope.go:117] "RemoveContainer" containerID="244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"
Jul 14 18:00:51.445633 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:00:51.445614    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8\": container with ID starting with 244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8 not found: ID does not exist" containerID="244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"
Jul 14 18:00:51.445688 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.445641    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8"} err="failed to get container status \"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8\": rpc error: code = NotFound desc = could not find container \"244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8\": container with ID starting with 244d8d677f05b93929c164700ebf387ceeca98592248c4363d25ca73244ee9f8 not found: ID does not exist"
Jul 14 18:00:51.460156 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.460131    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 18:00:51.465076 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:51.465059    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cf667b8ff-2t4r6"]
Jul 14 18:00:52.318191 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:00:52.318152    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" path="/var/lib/kubelet/pods/581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad/volumes"
Jul 14 18:01:07.046409 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:01:07.046378    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:01:07.061689 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:01:07.061668    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:01:07.503396 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:01:07.503368    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 14 18:03:20.058470 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.058439    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"]
Jul 14 18:03:20.058991 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.058712    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" containerName="console"
Jul 14 18:03:20.058991 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.058723    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" containerName="console"
Jul 14 18:03:20.058991 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.058771    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="581d2e50-d9ce-4d5b-98bf-3a34f2d4d7ad" containerName="console"
Jul 14 18:03:20.060440 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.060425    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.064535 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.064513    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 14 18:03:20.064693 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.064604    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-774rz\""
Jul 14 18:03:20.064693 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.064637    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 14 18:03:20.080432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.080403    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"]
Jul 14 18:03:20.206443 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.206419    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jhsg\" (UniqueName: \"kubernetes.io/projected/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-kube-api-access-7jhsg\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.206543 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.206448    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.307374 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.307336    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jhsg\" (UniqueName: \"kubernetes.io/projected/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-kube-api-access-7jhsg\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.307374 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.307376    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.307729 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.307712    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-tmp\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.317387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.317328    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jhsg\" (UniqueName: \"kubernetes.io/projected/a2c1a031-1ba7-4e94-aa48-9189390ff2fe-kube-api-access-7jhsg\") pod \"openshift-lws-operator-fd8ccff4c-w9fgk\" (UID: \"a2c1a031-1ba7-4e94-aa48-9189390ff2fe\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.376598 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.376561    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"
Jul 14 18:03:20.506094 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.506064    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk"]
Jul 14 18:03:20.511121 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:03:20.511095    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2c1a031_1ba7_4e94_aa48_9189390ff2fe.slice/crio-6e5cf228105e01ddc0eb132e713a4d8e34ef459c50750cb1794025ecaef231ed WatchSource:0}: Error finding container 6e5cf228105e01ddc0eb132e713a4d8e34ef459c50750cb1794025ecaef231ed: Status 404 returned error can't find the container with id 6e5cf228105e01ddc0eb132e713a4d8e34ef459c50750cb1794025ecaef231ed
Jul 14 18:03:20.955171 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:20.955140    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk" event={"ID":"a2c1a031-1ba7-4e94-aa48-9189390ff2fe","Type":"ContainerStarted","Data":"6e5cf228105e01ddc0eb132e713a4d8e34ef459c50750cb1794025ecaef231ed"}
Jul 14 18:03:23.967356 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:23.967319    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk" event={"ID":"a2c1a031-1ba7-4e94-aa48-9189390ff2fe","Type":"ContainerStarted","Data":"aa88ec723623b70f744818e022c1c01dfdc30d9a210e1a190b898b6f5b2b1096"}
Jul 14 18:03:23.988916 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:23.988872    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-w9fgk" podStartSLOduration=1.5290591629999999 podStartE2EDuration="3.988858118s" totalImagesPullingTime="2.459798955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:03:23.987277449 +0000 UTC m=+522.188832592" watchObservedRunningTime="2026-07-14 18:03:23.988858118 +0000 UTC m=+522.190413259"
Jul 14 18:03:41.665959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.665876    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"]
Jul 14 18:03:41.674259 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.674235    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.688693 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.688674    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 14 18:03:41.688810 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.688701    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 14 18:03:41.688810 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.688745    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 14 18:03:41.689416 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.689401    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vfmqw\""
Jul 14 18:03:41.689779 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.689763    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 14 18:03:41.695122 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.695083    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"]
Jul 14 18:03:41.748917 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.748889    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.749005 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.748929    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z874f\" (UniqueName: \"kubernetes.io/projected/5a0506ca-5185-4654-844a-d3dcc95f4d41-kube-api-access-z874f\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.749005 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.748978    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.749077 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.749015    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.749077 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.749062    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850226 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850197    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850330 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850249    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z874f\" (UniqueName: \"kubernetes.io/projected/5a0506ca-5185-4654-844a-d3dcc95f4d41-kube-api-access-z874f\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850330 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850288    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850438 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850340    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850438 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850375    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.850811 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.850783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-manifests\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.852756 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.852725    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5a0506ca-5185-4654-844a-d3dcc95f4d41-tmp\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.852973 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.852955    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-webhook-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.853019 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.852989    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5a0506ca-5185-4654-844a-d3dcc95f4d41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.865696 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.865678    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z874f\" (UniqueName: \"kubernetes.io/projected/5a0506ca-5185-4654-844a-d3dcc95f4d41-kube-api-access-z874f\") pod \"opendatahub-operator-controller-manager-67c464fdbc-2tr6j\" (UID: \"5a0506ca-5185-4654-844a-d3dcc95f4d41\") " pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:41.984050 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:41.983998    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:42.130058 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:42.130037    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"]
Jul 14 18:03:42.131862 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:03:42.131836    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a0506ca_5185_4654_844a_d3dcc95f4d41.slice/crio-8bd5704804a62e6237fc52eba8baf8d7b35a46bda1f99d67788ed0ca6f3f74b0 WatchSource:0}: Error finding container 8bd5704804a62e6237fc52eba8baf8d7b35a46bda1f99d67788ed0ca6f3f74b0: Status 404 returned error can't find the container with id 8bd5704804a62e6237fc52eba8baf8d7b35a46bda1f99d67788ed0ca6f3f74b0
Jul 14 18:03:43.036374 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:43.036340    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j" event={"ID":"5a0506ca-5185-4654-844a-d3dcc95f4d41","Type":"ContainerStarted","Data":"8bd5704804a62e6237fc52eba8baf8d7b35a46bda1f99d67788ed0ca6f3f74b0"}
Jul 14 18:03:48.060164 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:48.060085    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j" event={"ID":"5a0506ca-5185-4654-844a-d3dcc95f4d41","Type":"ContainerStarted","Data":"426988fa5d6a6062369a7e201af1ef31f05d2eb690558d384f2fff8a28ba77fd"}
Jul 14 18:03:49.065743 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:49.065711    2556 generic.go:356] "Generic (PLEG): container finished" podID="5a0506ca-5185-4654-844a-d3dcc95f4d41" containerID="426988fa5d6a6062369a7e201af1ef31f05d2eb690558d384f2fff8a28ba77fd" exitCode=0
Jul 14 18:03:49.066050 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:49.065766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j" event={"ID":"5a0506ca-5185-4654-844a-d3dcc95f4d41","Type":"ContainerDied","Data":"426988fa5d6a6062369a7e201af1ef31f05d2eb690558d384f2fff8a28ba77fd"}
Jul 14 18:03:50.071625 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:50.071567    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j" event={"ID":"5a0506ca-5185-4654-844a-d3dcc95f4d41","Type":"ContainerStarted","Data":"a4fbf6437193f4220d5a1de755921ac1286ddb805c265d290f9ac7f82c5d4382"}
Jul 14 18:03:50.072039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:50.071731    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:03:50.139192 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:03:50.139128    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j" podStartSLOduration=2.031919033 podStartE2EDuration="9.139112687s" totalImagesPullingTime="6.080299924s" totalInitContainerRuntime="1.02689373s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:03:41 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:03:50.135473979 +0000 UTC m=+548.337029121" watchObservedRunningTime="2026-07-14 18:03:50.139112687 +0000 UTC m=+548.340667925"
Jul 14 18:04:01.078772 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:01.078736    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67c464fdbc-2tr6j"
Jul 14 18:04:06.724995 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.724961    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"]
Jul 14 18:04:06.727862 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.727845    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.731056 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.730999    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 14 18:04:06.731267 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.731248    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 14 18:04:06.731982 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.731965    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wtbc8\""
Jul 14 18:04:06.746924 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.746906    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"]
Jul 14 18:04:06.824166 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.824142    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tls-certs\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.824242 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.824191    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l92x\" (UniqueName: \"kubernetes.io/projected/0cbd0de3-08a3-407b-b48c-3fa52677c55b-kube-api-access-6l92x\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.824289 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.824247    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tmp\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.924678 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.924649    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tls-certs\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.924839 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.924702    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l92x\" (UniqueName: \"kubernetes.io/projected/0cbd0de3-08a3-407b-b48c-3fa52677c55b-kube-api-access-6l92x\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.924839 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.924739    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tmp\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.927186 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.927163    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tmp\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.927371 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.927353    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbd0de3-08a3-407b-b48c-3fa52677c55b-tls-certs\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:06.938017 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:06.937987    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l92x\" (UniqueName: \"kubernetes.io/projected/0cbd0de3-08a3-407b-b48c-3fa52677c55b-kube-api-access-6l92x\") pod \"kube-auth-proxy-5979cf877b-pv9n9\" (UID: \"0cbd0de3-08a3-407b-b48c-3fa52677c55b\") " pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:07.036763 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:07.036714    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"
Jul 14 18:04:07.176480 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:07.176424    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9"]
Jul 14 18:04:07.178626 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:04:07.178577    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cbd0de3_08a3_407b_b48c_3fa52677c55b.slice/crio-7e382173048bc01f255ad18ac5579f9be743be63adfcc6d2926e930fe1e86824 WatchSource:0}: Error finding container 7e382173048bc01f255ad18ac5579f9be743be63adfcc6d2926e930fe1e86824: Status 404 returned error can't find the container with id 7e382173048bc01f255ad18ac5579f9be743be63adfcc6d2926e930fe1e86824
Jul 14 18:04:08.149684 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:08.149642    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9" event={"ID":"0cbd0de3-08a3-407b-b48c-3fa52677c55b","Type":"ContainerStarted","Data":"7e382173048bc01f255ad18ac5579f9be743be63adfcc6d2926e930fe1e86824"}
Jul 14 18:04:11.163531 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:11.163497    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9" event={"ID":"0cbd0de3-08a3-407b-b48c-3fa52677c55b","Type":"ContainerStarted","Data":"c40e24316ebe0dba5fa4bd9f8d7ec220371f6cf3cd3126619a6d66fced4b54bb"}
Jul 14 18:04:11.185932 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:11.185881    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5979cf877b-pv9n9" podStartSLOduration=1.657709787 podStartE2EDuration="5.185865407s" totalImagesPullingTime="3.52815562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:11.185672572 +0000 UTC m=+569.387227716" watchObservedRunningTime="2026-07-14 18:04:11.185865407 +0000 UTC m=+569.387420550"
Jul 14 18:04:15.824843 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.824810    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"]
Jul 14 18:04:15.828009 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.827993    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.837798 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.837777    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-k6glj\""
Jul 14 18:04:15.837912 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.837853    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 14 18:04:15.837979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.837918    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 14 18:04:15.838121 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.838107    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 14 18:04:15.861636 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.861610    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"]
Jul 14 18:04:15.888658 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.888639    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.888742 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.888696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f98f\" (UniqueName: \"kubernetes.io/projected/213ec993-5513-469c-b459-d56d4ae5e7f1-kube-api-access-6f98f\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.888742 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.888715    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/213ec993-5513-469c-b459-d56d4ae5e7f1-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.888819 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.888749    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.989615 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.989576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.989703 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.989646    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6f98f\" (UniqueName: \"kubernetes.io/projected/213ec993-5513-469c-b459-d56d4ae5e7f1-kube-api-access-6f98f\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.989703 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.989664    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/213ec993-5513-469c-b459-d56d4ae5e7f1-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.989703 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.989680    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.990327 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.990307    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/213ec993-5513-469c-b459-d56d4ae5e7f1-manager-config\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.992153 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.992125    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:15.992153 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:15.992149    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/213ec993-5513-469c-b459-d56d4ae5e7f1-metrics-cert\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:16.017991 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:16.017962    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f98f\" (UniqueName: \"kubernetes.io/projected/213ec993-5513-469c-b459-d56d4ae5e7f1-kube-api-access-6f98f\") pod \"lws-controller-manager-57cf6fdf7d-s7pj2\" (UID: \"213ec993-5513-469c-b459-d56d4ae5e7f1\") " pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:16.136449 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:16.136430    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:16.272771 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:16.272748    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"]
Jul 14 18:04:16.275073 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:04:16.275046    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod213ec993_5513_469c_b459_d56d4ae5e7f1.slice/crio-f0988be149d32881d89a7e875b79f47e09737e26c2baa1bac93dc94890ea4015 WatchSource:0}: Error finding container f0988be149d32881d89a7e875b79f47e09737e26c2baa1bac93dc94890ea4015: Status 404 returned error can't find the container with id f0988be149d32881d89a7e875b79f47e09737e26c2baa1bac93dc94890ea4015
Jul 14 18:04:17.189230 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:17.189154    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2" event={"ID":"213ec993-5513-469c-b459-d56d4ae5e7f1","Type":"ContainerStarted","Data":"f0988be149d32881d89a7e875b79f47e09737e26c2baa1bac93dc94890ea4015"}
Jul 14 18:04:18.200446 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:18.200407    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2" event={"ID":"213ec993-5513-469c-b459-d56d4ae5e7f1","Type":"ContainerStarted","Data":"9dc11fa6698d47cb4fb372ad460974ff3369e2ed84308c71345addc7e819e184"}
Jul 14 18:04:18.200871 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:18.200545    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:18.228051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:18.228012    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2" podStartSLOduration=1.9780294189999998 podStartE2EDuration="3.228001472s" totalImagesPullingTime="1.249972053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:18.227840687 +0000 UTC m=+576.429395829" watchObservedRunningTime="2026-07-14 18:04:18.228001472 +0000 UTC m=+576.429556612"
Jul 14 18:04:29.206995 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:29.206967    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-57cf6fdf7d-s7pj2"
Jul 14 18:04:42.168141 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:42.168112    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:04:42.168536 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:42.168112    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:04:42.175147 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:42.175126    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:04:42.175498 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:04:42.175472    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:06:06.342310 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.342271    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:06:06.344658 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.344636    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.347695 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.347659    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zbch6\""
Jul 14 18:06:06.347857 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.347751    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 14 18:06:06.348006 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.347990    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 14 18:06:06.359044 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.359020    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkwcl\" (UniqueName: \"kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.359139 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.359063    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.378815 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.378789    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:06:06.460247 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.460224    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkwcl\" (UniqueName: \"kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.460340 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.460261    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.460664 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.460648    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.476516 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.476485    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkwcl\" (UniqueName: \"kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl\") pod \"kuadrant-operator-controller-manager-86878c769f-zwwfz\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.655220 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.655197    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:06.796194 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:06:06.796168    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a5f3448_8750_4425_9df5_8f85cc546110.slice/crio-218d88b14b623fd7c7b32bc3b5a5196b126ea8ef6f769322f4bb79a8362633b2 WatchSource:0}: Error finding container 218d88b14b623fd7c7b32bc3b5a5196b126ea8ef6f769322f4bb79a8362633b2: Status 404 returned error can't find the container with id 218d88b14b623fd7c7b32bc3b5a5196b126ea8ef6f769322f4bb79a8362633b2
Jul 14 18:06:06.798611 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.798572    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 18:06:06.801057 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:06.801034    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:06:07.645540 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:07.645504    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" event={"ID":"0a5f3448-8750-4425-9df5-8f85cc546110","Type":"ContainerStarted","Data":"218d88b14b623fd7c7b32bc3b5a5196b126ea8ef6f769322f4bb79a8362633b2"}
Jul 14 18:06:12.669277 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:12.669241    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" event={"ID":"0a5f3448-8750-4425-9df5-8f85cc546110","Type":"ContainerStarted","Data":"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"}
Jul 14 18:06:12.669684 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:12.669381    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:23.675103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:23.675072    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:06:23.705329 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:23.705282    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" podStartSLOduration=12.370999276 podStartE2EDuration="17.705266814s" totalImagesPullingTime="5.334267538s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:12.702930943 +0000 UTC m=+690.904486084" watchObservedRunningTime="2026-07-14 18:06:23.705266814 +0000 UTC m=+701.906821954"
Jul 14 18:06:47.043564 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.043483    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:06:47.051880 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.051851    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:06:47.055410 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.055390    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-l7nd9\""
Jul 14 18:06:47.088312 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.088286    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:06:47.154090 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.154066    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w928j\" (UniqueName: \"kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j\") pod \"authorino-75d556bd77-ph54h\" (UID: \"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8\") " pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:06:47.255390 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.255364    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w928j\" (UniqueName: \"kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j\") pod \"authorino-75d556bd77-ph54h\" (UID: \"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8\") " pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:06:47.275937 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.275916    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w928j\" (UniqueName: \"kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j\") pod \"authorino-75d556bd77-ph54h\" (UID: \"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8\") " pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:06:47.360867 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.360849    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:06:47.499933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.499902    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:06:47.502851 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:06:47.502825    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84a59a50_1a7e_43ca_b76e_ebeb54bc47f8.slice/crio-3da2a3a63ab5c78d0a26c6a5aa0ad02ae6ffb0709ca301b97a7e35489f834aac WatchSource:0}: Error finding container 3da2a3a63ab5c78d0a26c6a5aa0ad02ae6ffb0709ca301b97a7e35489f834aac: Status 404 returned error can't find the container with id 3da2a3a63ab5c78d0a26c6a5aa0ad02ae6ffb0709ca301b97a7e35489f834aac
Jul 14 18:06:47.818848 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:47.818760    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ph54h" event={"ID":"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8","Type":"ContainerStarted","Data":"3da2a3a63ab5c78d0a26c6a5aa0ad02ae6ffb0709ca301b97a7e35489f834aac"}
Jul 14 18:06:50.832963 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:50.832937    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ph54h" event={"ID":"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8","Type":"ContainerStarted","Data":"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"}
Jul 14 18:06:50.849674 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:06:50.849631    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-ph54h" podStartSLOduration=0.588183384 podStartE2EDuration="3.849616207s" totalImagesPullingTime="3.261432823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:06:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:06:50.847977286 +0000 UTC m=+729.049532427" watchObservedRunningTime="2026-07-14 18:06:50.849616207 +0000 UTC m=+729.051171348"
Jul 14 18:07:14.922753 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:14.922719    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:07:14.925872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:14.925853    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:14.928729 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:14.928704    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 14 18:07:14.945615 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:14.945566    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:07:15.082928 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.082886    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.083111 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.082933    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpqd7\" (UniqueName: \"kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.183817 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.183721    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.183817 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.183769    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpqd7\" (UniqueName: \"kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.186408 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.186386    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.195942 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.195911    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpqd7\" (UniqueName: \"kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7\") pod \"authorino-7544f857b4-q8hrd\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") " pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.236731 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.236698    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:07:15.382293 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.382267    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:07:15.384051 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:07:15.384024    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7cc23760_69c0_41ee_b7df_77b5fc769194.slice/crio-56d59309cb97f95c8c08b0952719fee89d0e69069f7c3570a02880e96ffe2ab0 WatchSource:0}: Error finding container 56d59309cb97f95c8c08b0952719fee89d0e69069f7c3570a02880e96ffe2ab0: Status 404 returned error can't find the container with id 56d59309cb97f95c8c08b0952719fee89d0e69069f7c3570a02880e96ffe2ab0
Jul 14 18:07:15.945558 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.945513    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7544f857b4-q8hrd" event={"ID":"7cc23760-69c0-41ee-b7df-77b5fc769194","Type":"ContainerStarted","Data":"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"}
Jul 14 18:07:15.946034 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.945569    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7544f857b4-q8hrd" event={"ID":"7cc23760-69c0-41ee-b7df-77b5fc769194","Type":"ContainerStarted","Data":"56d59309cb97f95c8c08b0952719fee89d0e69069f7c3570a02880e96ffe2ab0"}
Jul 14 18:07:15.996628 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:15.996538    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7544f857b4-q8hrd" podStartSLOduration=1.550029216 podStartE2EDuration="1.996518491s" totalImagesPullingTime="446.489275ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:15.978078928 +0000 UTC m=+754.179634068" watchObservedRunningTime="2026-07-14 18:07:15.996518491 +0000 UTC m=+754.198073632"
Jul 14 18:07:16.059772 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.059733    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:07:16.059945 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.059914    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-ph54h" podUID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" containerName="authorino" containerID="cri-o://40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275" gracePeriod=30
Jul 14 18:07:16.312334 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.312304    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:07:16.393431 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.393395    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w928j\" (UniqueName: \"kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j\") pod \"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8\" (UID: \"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8\") "
Jul 14 18:07:16.395792 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.395758    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j" (OuterVolumeSpecName: "kube-api-access-w928j") pod "84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" (UID: "84a59a50-1a7e-43ca-b76e-ebeb54bc47f8"). InnerVolumeSpecName "kube-api-access-w928j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:07:16.494419 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.494333    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w928j\" (UniqueName: \"kubernetes.io/projected/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8-kube-api-access-w928j\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:07:16.950950 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.950912    2556 generic.go:356] "Generic (PLEG): container finished" podID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" containerID="40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275" exitCode=0
Jul 14 18:07:16.951398 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.950961    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-ph54h"
Jul 14 18:07:16.951398 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.950998    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ph54h" event={"ID":"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8","Type":"ContainerDied","Data":"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"}
Jul 14 18:07:16.951398 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.951045    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-ph54h" event={"ID":"84a59a50-1a7e-43ca-b76e-ebeb54bc47f8","Type":"ContainerDied","Data":"3da2a3a63ab5c78d0a26c6a5aa0ad02ae6ffb0709ca301b97a7e35489f834aac"}
Jul 14 18:07:16.951398 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.951063    2556 scope.go:117] "RemoveContainer" containerID="40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"
Jul 14 18:07:16.959707 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.959685    2556 scope.go:117] "RemoveContainer" containerID="40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"
Jul 14 18:07:16.959988 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:07:16.959968    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275\": container with ID starting with 40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275 not found: ID does not exist" containerID="40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"
Jul 14 18:07:16.960062 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.959999    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275"} err="failed to get container status \"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275\": rpc error: code = NotFound desc = could not find container \"40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275\": container with ID starting with 40391496e98c819513ce64a8fdfcc1b8f592fc4418439077852b06826845f275 not found: ID does not exist"
Jul 14 18:07:16.990407 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.990367    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:07:16.997334 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:16.997309    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-ph54h"]
Jul 14 18:07:18.319228 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:07:18.319192    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" path="/var/lib/kubelet/pods/84a59a50-1a7e-43ca-b76e-ebeb54bc47f8/volumes"
Jul 14 18:08:33.100926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.100889    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"]
Jul 14 18:08:33.101314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.101218    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" containerName="authorino"
Jul 14 18:08:33.101314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.101229    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" containerName="authorino"
Jul 14 18:08:33.101314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.101287    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="84a59a50-1a7e-43ca-b76e-ebeb54bc47f8" containerName="authorino"
Jul 14 18:08:33.104341 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.104319    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.112008 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.111988    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 14 18:08:33.116365 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.116346    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 14 18:08:33.117524 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.117509    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 14 18:08:33.118481 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.118466    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6f4ng\""
Jul 14 18:08:33.139081 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.139056    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"]
Jul 14 18:08:33.251975 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.251953    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5015c76-f0f4-4b69-8778-491600d05414-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.252095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.251988    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.252095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.252019    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.252095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.252043    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ls2r\" (UniqueName: \"kubernetes.io/projected/e5015c76-f0f4-4b69-8778-491600d05414-kube-api-access-4ls2r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.252095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.252078    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.252240 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.252101    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352542 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352482    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352542 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352525    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352717 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352555    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ls2r\" (UniqueName: \"kubernetes.io/projected/e5015c76-f0f4-4b69-8778-491600d05414-kube-api-access-4ls2r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352717 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352603    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352717 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352632    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352717 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352682    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5015c76-f0f4-4b69-8778-491600d05414-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.352972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352951    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.353015 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.352996    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.353049 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.353028    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.355013 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.354992    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e5015c76-f0f4-4b69-8778-491600d05414-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.355088 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.355071    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e5015c76-f0f4-4b69-8778-491600d05414-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.374296 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.374269    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ls2r\" (UniqueName: \"kubernetes.io/projected/e5015c76-f0f4-4b69-8778-491600d05414-kube-api-access-4ls2r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gkhj5\" (UID: \"e5015c76-f0f4-4b69-8778-491600d05414\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.414143 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.414123    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:33.576215 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:33.576191    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"]
Jul 14 18:08:33.578567 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:08:33.578544    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5015c76_f0f4_4b69_8778_491600d05414.slice/crio-9a2d61f9173e43272b0334f955567b1761a8f9be6beed4341ec462b18647d992 WatchSource:0}: Error finding container 9a2d61f9173e43272b0334f955567b1761a8f9be6beed4341ec462b18647d992: Status 404 returned error can't find the container with id 9a2d61f9173e43272b0334f955567b1761a8f9be6beed4341ec462b18647d992
Jul 14 18:08:34.292136 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:34.292102    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5" event={"ID":"e5015c76-f0f4-4b69-8778-491600d05414","Type":"ContainerStarted","Data":"9a2d61f9173e43272b0334f955567b1761a8f9be6beed4341ec462b18647d992"}
Jul 14 18:08:35.297666 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:35.297631    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5" event={"ID":"e5015c76-f0f4-4b69-8778-491600d05414","Type":"ContainerStarted","Data":"2772366693c5e4c9ebe09e66fd3728ed761195c9d7b513cbabc4a972df201fe7"}
Jul 14 18:08:35.322261 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:35.322204    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5" podStartSLOduration=1.216487863 podStartE2EDuration="2.322189049s" totalImagesPullingTime="1.105701186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:35.321150598 +0000 UTC m=+833.522705740" watchObservedRunningTime="2026-07-14 18:08:35.322189049 +0000 UTC m=+833.523744189"
Jul 14 18:08:43.414459 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:43.414428    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:43.414916 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:43.414486    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:43.426341 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:43.426322    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:08:44.343513 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:08:44.343479    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gkhj5"
Jul 14 18:09:00.998508 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:00.998463    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"]
Jul 14 18:09:01.002041 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.001981    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.006932 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.006901    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 14 18:09:01.018765 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.018738    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"]
Jul 14 18:09:01.062161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062118    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/798241e5-e7cb-43b4-8e61-54a6dca46c58-kube-api-access-nv2sf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.062161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062163    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.062387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062229    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.062387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062356    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.062493 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062392    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.062493 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.062442    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798241e5-e7cb-43b4-8e61-54a6dca46c58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.163775 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163740    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.163930 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163781    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.163930 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163821    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798241e5-e7cb-43b4-8e61-54a6dca46c58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.163930 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163860    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/798241e5-e7cb-43b4-8e61-54a6dca46c58-kube-api-access-nv2sf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.163930 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163885    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.164137 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.163946    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.164206 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.164162    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.164274 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.164214    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.164358 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.164337    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.166247 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.166208    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/798241e5-e7cb-43b4-8e61-54a6dca46c58-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.166453 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.166431    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/798241e5-e7cb-43b4-8e61-54a6dca46c58-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.188450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.188419    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2sf\" (UniqueName: \"kubernetes.io/projected/798241e5-e7cb-43b4-8e61-54a6dca46c58-kube-api-access-nv2sf\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2\" (UID: \"798241e5-e7cb-43b4-8e61-54a6dca46c58\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.311325 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.311237    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:01.533301 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:01.533268    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"]
Jul 14 18:09:01.537791 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:09:01.537758    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod798241e5_e7cb_43b4_8e61_54a6dca46c58.slice/crio-14e1bdc9dbe40d314294bb2cbaae9b98b23589f204193d0e354a3d4b02139eb0 WatchSource:0}: Error finding container 14e1bdc9dbe40d314294bb2cbaae9b98b23589f204193d0e354a3d4b02139eb0: Status 404 returned error can't find the container with id 14e1bdc9dbe40d314294bb2cbaae9b98b23589f204193d0e354a3d4b02139eb0
Jul 14 18:09:02.425822 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:02.425777    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2" event={"ID":"798241e5-e7cb-43b4-8e61-54a6dca46c58","Type":"ContainerStarted","Data":"737c038c01a7a769ad9105627cbaac553a51f3085805486b9d77bdb08cb972cf"}
Jul 14 18:09:02.426212 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:02.425828    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2" event={"ID":"798241e5-e7cb-43b4-8e61-54a6dca46c58","Type":"ContainerStarted","Data":"14e1bdc9dbe40d314294bb2cbaae9b98b23589f204193d0e354a3d4b02139eb0"}
Jul 14 18:09:02.450518 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:02.450467    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2" podStartSLOduration=2.274260011 podStartE2EDuration="2.450451728s" totalImagesPullingTime="176.191717ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:09:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:09:02.449609551 +0000 UTC m=+860.651164690" watchObservedRunningTime="2026-07-14 18:09:02.450451728 +0000 UTC m=+860.652006868"
Jul 14 18:09:11.312053 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:11.312008    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:11.312053 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:11.312065    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:11.324000 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:11.323980    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:11.474566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:11.474540    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2"
Jul 14 18:09:42.191815 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:42.191734    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:09:42.193268 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:42.193241    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:09:42.198996 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:42.198976    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:09:42.199884 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:09:42.199868    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:10:26.209779 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.209743    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d7d9c478f-znwrw"]
Jul 14 18:10:26.212855 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.212833    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.223762 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.223740    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d7d9c478f-znwrw"]
Jul 14 18:10:26.253380 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.253352    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00b6830b-9392-4d2d-8885-d8638a9b1889-tls-cert\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.253474 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.253397    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snf7d\" (UniqueName: \"kubernetes.io/projected/00b6830b-9392-4d2d-8885-d8638a9b1889-kube-api-access-snf7d\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.354008 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.353979    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snf7d\" (UniqueName: \"kubernetes.io/projected/00b6830b-9392-4d2d-8885-d8638a9b1889-kube-api-access-snf7d\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.354096 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.354086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00b6830b-9392-4d2d-8885-d8638a9b1889-tls-cert\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.356489 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.356469    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/00b6830b-9392-4d2d-8885-d8638a9b1889-tls-cert\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.362863 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.362842    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snf7d\" (UniqueName: \"kubernetes.io/projected/00b6830b-9392-4d2d-8885-d8638a9b1889-kube-api-access-snf7d\") pod \"authorino-7d7d9c478f-znwrw\" (UID: \"00b6830b-9392-4d2d-8885-d8638a9b1889\") " pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.521867 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.521812    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d7d9c478f-znwrw"
Jul 14 18:10:26.666114 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.666085    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d7d9c478f-znwrw"]
Jul 14 18:10:26.669598 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:10:26.669556    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00b6830b_9392_4d2d_8885_d8638a9b1889.slice/crio-4693d6f512d8743f2a537d9f1deba1a9ccb686f726919f8c956b5b1ffe37d40c WatchSource:0}: Error finding container 4693d6f512d8743f2a537d9f1deba1a9ccb686f726919f8c956b5b1ffe37d40c: Status 404 returned error can't find the container with id 4693d6f512d8743f2a537d9f1deba1a9ccb686f726919f8c956b5b1ffe37d40c
Jul 14 18:10:26.764875 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:26.764843    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d7d9c478f-znwrw" event={"ID":"00b6830b-9392-4d2d-8885-d8638a9b1889","Type":"ContainerStarted","Data":"4693d6f512d8743f2a537d9f1deba1a9ccb686f726919f8c956b5b1ffe37d40c"}
Jul 14 18:10:27.771130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:27.771090    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d7d9c478f-znwrw" event={"ID":"00b6830b-9392-4d2d-8885-d8638a9b1889","Type":"ContainerStarted","Data":"53d89ee4b3b98e58283f872cafd14bbfcd58a8ad7963bba9ffe61bfe14f4b8f7"}
Jul 14 18:10:27.817014 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:27.816949    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d7d9c478f-znwrw" podStartSLOduration=1.415252711 podStartE2EDuration="1.81692909s" totalImagesPullingTime="401.676379ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:10:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:10:27.816448784 +0000 UTC m=+946.018003925" watchObservedRunningTime="2026-07-14 18:10:27.81692909 +0000 UTC m=+946.018484230"
Jul 14 18:10:27.875268 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:27.875232    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:10:27.875475 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:27.875448    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7544f857b4-q8hrd" podUID="7cc23760-69c0-41ee-b7df-77b5fc769194" containerName="authorino" containerID="cri-o://fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f" gracePeriod=30
Jul 14 18:10:28.138431 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.138404    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:10:28.168788 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.168762    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cpqd7\" (UniqueName: \"kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7\") pod \"7cc23760-69c0-41ee-b7df-77b5fc769194\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") "
Jul 14 18:10:28.168942 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.168813    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert\") pod \"7cc23760-69c0-41ee-b7df-77b5fc769194\" (UID: \"7cc23760-69c0-41ee-b7df-77b5fc769194\") "
Jul 14 18:10:28.171367 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.171324    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7" (OuterVolumeSpecName: "kube-api-access-cpqd7") pod "7cc23760-69c0-41ee-b7df-77b5fc769194" (UID: "7cc23760-69c0-41ee-b7df-77b5fc769194"). InnerVolumeSpecName "kube-api-access-cpqd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:10:28.180197 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.180173    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "7cc23760-69c0-41ee-b7df-77b5fc769194" (UID: "7cc23760-69c0-41ee-b7df-77b5fc769194"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:10:28.269803 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.269771    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cpqd7\" (UniqueName: \"kubernetes.io/projected/7cc23760-69c0-41ee-b7df-77b5fc769194-kube-api-access-cpqd7\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:10:28.269803 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.269800    2556 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/7cc23760-69c0-41ee-b7df-77b5fc769194-tls-cert\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:10:28.777621 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.777572    2556 generic.go:356] "Generic (PLEG): container finished" podID="7cc23760-69c0-41ee-b7df-77b5fc769194" containerID="fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f" exitCode=0
Jul 14 18:10:28.778001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.777626    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7544f857b4-q8hrd"
Jul 14 18:10:28.778001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.777661    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7544f857b4-q8hrd" event={"ID":"7cc23760-69c0-41ee-b7df-77b5fc769194","Type":"ContainerDied","Data":"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"}
Jul 14 18:10:28.778001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.777706    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7544f857b4-q8hrd" event={"ID":"7cc23760-69c0-41ee-b7df-77b5fc769194","Type":"ContainerDied","Data":"56d59309cb97f95c8c08b0952719fee89d0e69069f7c3570a02880e96ffe2ab0"}
Jul 14 18:10:28.778001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.777730    2556 scope.go:117] "RemoveContainer" containerID="fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"
Jul 14 18:10:28.786604 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.786557    2556 scope.go:117] "RemoveContainer" containerID="fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"
Jul 14 18:10:28.786904 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:10:28.786881    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f\": container with ID starting with fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f not found: ID does not exist" containerID="fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"
Jul 14 18:10:28.786985 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.786913    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f"} err="failed to get container status \"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f\": rpc error: code = NotFound desc = could not find container \"fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f\": container with ID starting with fef869e06fab9a9d80acccda88646af403d027f67fee478f5cfa7e802200c72f not found: ID does not exist"
Jul 14 18:10:28.812486 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.812458    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:10:28.821871 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:28.821839    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7544f857b4-q8hrd"]
Jul 14 18:10:30.319819 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:10:30.319786    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7cc23760-69c0-41ee-b7df-77b5fc769194" path="/var/lib/kubelet/pods/7cc23760-69c0-41ee-b7df-77b5fc769194/volumes"
Jul 14 18:14:42.215223 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:14:42.215192    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:14:42.217054 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:14:42.217027    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:14:42.221528 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:14:42.221505    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:14:42.223385 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:14:42.223368    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:16:45.696162 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:45.696126    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:16:45.698570 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:45.696381    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" podUID="0a5f3448-8750-4425-9df5-8f85cc546110" containerName="manager" containerID="cri-o://6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd" gracePeriod=10
Jul 14 18:16:49.047264 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.047243    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:16:49.108535 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.108507    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume\") pod \"0a5f3448-8750-4425-9df5-8f85cc546110\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") "
Jul 14 18:16:49.108686 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.108560    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkwcl\" (UniqueName: \"kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl\") pod \"0a5f3448-8750-4425-9df5-8f85cc546110\" (UID: \"0a5f3448-8750-4425-9df5-8f85cc546110\") "
Jul 14 18:16:49.108859 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.108837    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0a5f3448-8750-4425-9df5-8f85cc546110" (UID: "0a5f3448-8750-4425-9df5-8f85cc546110"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:16:49.110657 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.110636    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl" (OuterVolumeSpecName: "kube-api-access-hkwcl") pod "0a5f3448-8750-4425-9df5-8f85cc546110" (UID: "0a5f3448-8750-4425-9df5-8f85cc546110"). InnerVolumeSpecName "kube-api-access-hkwcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:16:49.209453 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.209432    2556 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0a5f3448-8750-4425-9df5-8f85cc546110-extensions-socket-volume\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:16:49.209453 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.209453    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkwcl\" (UniqueName: \"kubernetes.io/projected/0a5f3448-8750-4425-9df5-8f85cc546110-kube-api-access-hkwcl\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:16:49.349930 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.349905    2556 generic.go:356] "Generic (PLEG): container finished" podID="0a5f3448-8750-4425-9df5-8f85cc546110" containerID="6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd" exitCode=0
Jul 14 18:16:49.350036 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.349960    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"
Jul 14 18:16:49.350075 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.349961    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" event={"ID":"0a5f3448-8750-4425-9df5-8f85cc546110","Type":"ContainerDied","Data":"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"}
Jul 14 18:16:49.350075 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.350070    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz" event={"ID":"0a5f3448-8750-4425-9df5-8f85cc546110","Type":"ContainerDied","Data":"218d88b14b623fd7c7b32bc3b5a5196b126ea8ef6f769322f4bb79a8362633b2"}
Jul 14 18:16:49.350145 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.350091    2556 scope.go:117] "RemoveContainer" containerID="6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"
Jul 14 18:16:49.359780 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.359762    2556 scope.go:117] "RemoveContainer" containerID="6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"
Jul 14 18:16:49.360061 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:16:49.360039    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd\": container with ID starting with 6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd not found: ID does not exist" containerID="6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"
Jul 14 18:16:49.360100 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.360070    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd"} err="failed to get container status \"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd\": rpc error: code = NotFound desc = could not find container \"6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd\": container with ID starting with 6ac76cf02f22f7eb08ecd4debfdf4702180b99b4417fa4f21d3d2ec7b35016dd not found: ID does not exist"
Jul 14 18:16:49.376275 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.376253    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:16:49.386450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:49.386431    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-zwwfz"]
Jul 14 18:16:50.319373 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:16:50.319335    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a5f3448-8750-4425-9df5-8f85cc546110" path="/var/lib/kubelet/pods/0a5f3448-8750-4425-9df5-8f85cc546110/volumes"
Jul 14 18:17:46.975286 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975246    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"]
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975543    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7cc23760-69c0-41ee-b7df-77b5fc769194" containerName="authorino"
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975554    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="7cc23760-69c0-41ee-b7df-77b5fc769194" containerName="authorino"
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975576    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a5f3448-8750-4425-9df5-8f85cc546110" containerName="manager"
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975594    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a5f3448-8750-4425-9df5-8f85cc546110" containerName="manager"
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975644    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a5f3448-8750-4425-9df5-8f85cc546110" containerName="manager"
Jul 14 18:17:46.975710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.975652    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="7cc23760-69c0-41ee-b7df-77b5fc769194" containerName="authorino"
Jul 14 18:17:46.978781 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.978763    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:46.982914 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:46.982892    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-zbch6\""
Jul 14 18:17:47.001687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.001666    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"]
Jul 14 18:17:47.130909 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.130887    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6379105-89d8-49af-ae2a-933e595e39e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.131011 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.130926    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lms8x\" (UniqueName: \"kubernetes.io/projected/e6379105-89d8-49af-ae2a-933e595e39e2-kube-api-access-lms8x\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.231730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.231680    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6379105-89d8-49af-ae2a-933e595e39e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.231730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.231719    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lms8x\" (UniqueName: \"kubernetes.io/projected/e6379105-89d8-49af-ae2a-933e595e39e2-kube-api-access-lms8x\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.232035 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.232017    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e6379105-89d8-49af-ae2a-933e595e39e2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.243519 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.243491    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lms8x\" (UniqueName: \"kubernetes.io/projected/e6379105-89d8-49af-ae2a-933e595e39e2-kube-api-access-lms8x\") pod \"kuadrant-operator-controller-manager-86878c769f-m9pm4\" (UID: \"e6379105-89d8-49af-ae2a-933e595e39e2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.288356 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.288334    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.431161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.431131    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"]
Jul 14 18:17:47.434337 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:17:47.434312    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6379105_89d8_49af_ae2a_933e595e39e2.slice/crio-39cbbc17b02cad8a3a67b5dffacd2e3529d607c76570f5f0f6aea5b4240410a3 WatchSource:0}: Error finding container 39cbbc17b02cad8a3a67b5dffacd2e3529d607c76570f5f0f6aea5b4240410a3: Status 404 returned error can't find the container with id 39cbbc17b02cad8a3a67b5dffacd2e3529d607c76570f5f0f6aea5b4240410a3
Jul 14 18:17:47.583230 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.583204    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4" event={"ID":"e6379105-89d8-49af-ae2a-933e595e39e2","Type":"ContainerStarted","Data":"37e5f7b90f7ff9bb299589cef6df5c08f8114470be1d9848c8fdde1a1b5ac242"}
Jul 14 18:17:47.583390 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.583248    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4" event={"ID":"e6379105-89d8-49af-ae2a-933e595e39e2","Type":"ContainerStarted","Data":"39cbbc17b02cad8a3a67b5dffacd2e3529d607c76570f5f0f6aea5b4240410a3"}
Jul 14 18:17:47.583390 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.583306    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:17:47.615934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:47.615884    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4" podStartSLOduration=1.6158670910000001 podStartE2EDuration="1.615867091s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:17:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:17:47.613490981 +0000 UTC m=+1385.815046121" watchObservedRunningTime="2026-07-14 18:17:47.615867091 +0000 UTC m=+1385.817422232"
Jul 14 18:17:58.589721 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:17:58.589694    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m9pm4"
Jul 14 18:19:42.239219 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:19:42.239185    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:19:42.240638 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:19:42.240617    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:19:42.246313 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:19:42.246292    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:19:42.247693 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:19:42.247673    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:24:18.491618 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.491573    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:18.494451 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.494434    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:18.499234 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.499209    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\""
Jul 14 18:24:18.499638 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.499620    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:24:18.509947 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.509926    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:18.620163 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.620135    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9\" (UID: \"177b0101-0f24-4ea0-88b5-a700a3d56797\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:18.720947 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.720916    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9\" (UID: \"177b0101-0f24-4ea0-88b5-a700a3d56797\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:18.721470 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.721452    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9\" (UID: \"177b0101-0f24-4ea0-88b5-a700a3d56797\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:18.804110 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.804059    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:18.938314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.938284    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:18.941317 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:24:18.941287    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177b0101_0f24_4ea0_88b5_a700a3d56797.slice/crio-f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf WatchSource:0}: Error finding container f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf: Status 404 returned error can't find the container with id f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf
Jul 14 18:24:18.943348 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:18.943327    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 18:24:19.225121 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:19.225077    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" event={"ID":"177b0101-0f24-4ea0-88b5-a700a3d56797","Type":"ContainerStarted","Data":"f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf"}
Jul 14 18:24:22.240685 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:22.240652    2556 generic.go:356] "Generic (PLEG): container finished" podID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerID="55af58869564e554fc83fcca32bf3d23fe985ec1b997c192c3f6cc85bb9ef84d" exitCode=7
Jul 14 18:24:22.241062 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:22.240740    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" event={"ID":"177b0101-0f24-4ea0-88b5-a700a3d56797","Type":"ContainerDied","Data":"55af58869564e554fc83fcca32bf3d23fe985ec1b997c192c3f6cc85bb9ef84d"}
Jul 14 18:24:22.241062 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:22.240868    2556 scope.go:117] "RemoveContainer" containerID="55af58869564e554fc83fcca32bf3d23fe985ec1b997c192c3f6cc85bb9ef84d"
Jul 14 18:24:23.246456 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:23.246385    2556 generic.go:356] "Generic (PLEG): container finished" podID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerID="9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b" exitCode=7
Jul 14 18:24:23.246876 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:23.246460    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" event={"ID":"177b0101-0f24-4ea0-88b5-a700a3d56797","Type":"ContainerDied","Data":"9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b"}
Jul 14 18:24:23.246876 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:23.246501    2556 scope.go:117] "RemoveContainer" containerID="55af58869564e554fc83fcca32bf3d23fe985ec1b997c192c3f6cc85bb9ef84d"
Jul 14 18:24:23.246876 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:23.246763    2556 scope.go:117] "RemoveContainer" containerID="9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b"
Jul 14 18:24:23.247019 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:24:23.246964    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9_odh-ai-gateway-infra(177b0101-0f24-4ea0-88b5-a700a3d56797)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797"
Jul 14 18:24:24.252700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:24.252677    2556 scope.go:117] "RemoveContainer" containerID="9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b"
Jul 14 18:24:24.253040 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:24:24.252843    2556 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9_odh-ai-gateway-infra(177b0101-0f24-4ea0-88b5-a700a3d56797)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797"
Jul 14 18:24:35.313770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:35.313690    2556 scope.go:117] "RemoveContainer" containerID="9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b"
Jul 14 18:24:36.302841 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:36.302811    2556 generic.go:356] "Generic (PLEG): container finished" podID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerID="678e779264aff6203c073b1b2435b50020cec117921636da45d8510616b3f1c2" exitCode=0
Jul 14 18:24:36.302948 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:36.302888    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" event={"ID":"177b0101-0f24-4ea0-88b5-a700a3d56797","Type":"ContainerDied","Data":"678e779264aff6203c073b1b2435b50020cec117921636da45d8510616b3f1c2"}
Jul 14 18:24:36.302984 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:36.302949    2556 scope.go:117] "RemoveContainer" containerID="9be71c3361f86a1a605eff7c6217883780f8fd6104eab64bf65f727ed6a97d6b"
Jul 14 18:24:37.343561 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:37.343530    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:37.431183 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:37.431158    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:37.562173 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:37.562114    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca\") pod \"177b0101-0f24-4ea0-88b5-a700a3d56797\" (UID: \"177b0101-0f24-4ea0-88b5-a700a3d56797\") "
Jul 14 18:24:37.562444 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:37.562427    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "177b0101-0f24-4ea0-88b5-a700a3d56797" (UID: "177b0101-0f24-4ea0-88b5-a700a3d56797"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:24:37.663345 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:37.663320    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/177b0101-0f24-4ea0-88b5-a700a3d56797-maas-api-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:24:38.313490 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:38.313470    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"
Jul 14 18:24:38.318483 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:38.318459    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9" event={"ID":"177b0101-0f24-4ea0-88b5-a700a3d56797","Type":"ContainerDied","Data":"f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf"}
Jul 14 18:24:38.318607 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:38.318498    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f6960631a6f83d4691936ea304777076df6b899ab684497fe7812ebfa4bc53cf"
Jul 14 18:24:38.353385 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:38.353364    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:38.361126 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:38.361104    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hhlk9"]
Jul 14 18:24:40.319782 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:40.319743    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" path="/var/lib/kubelet/pods/177b0101-0f24-4ea0-88b5-a700a3d56797/volumes"
Jul 14 18:24:42.263439 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:42.263412    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:24:42.268540 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:42.268518    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:24:42.270215 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:42.270193    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:24:42.274802 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:24:42.274788    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:25:08.539165 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539129    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"]
Jul 14 18:25:08.539669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539534    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539553    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539600    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539608    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539881 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539685    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539881 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539698    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.539881 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.539706    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:08.544218 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.544197    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:08.548778 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.548759    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:25:08.548878 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.548761    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\""
Jul 14 18:25:08.555578 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.555556    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"]
Jul 14 18:25:08.684627 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.684602    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb\" (UID: \"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:08.785941 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.785917    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb\" (UID: \"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:08.786478 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.786458    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb\" (UID: \"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:08.853477 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.853457    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:08.987443 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:08.987411    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"]
Jul 14 18:25:08.989102 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:25:08.989076    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cf214c5_7bee_42c9_a0ed_d5a7f72bba3e.slice/crio-9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5 WatchSource:0}: Error finding container 9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5: Status 404 returned error can't find the container with id 9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5
Jul 14 18:25:09.436127 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:09.436067    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb" event={"ID":"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e","Type":"ContainerStarted","Data":"a76e5d9f9b466b9f645b01fe353a15d21042d708a507d4874c1cbe9d92d12be8"}
Jul 14 18:25:09.436127 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:09.436116    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb" event={"ID":"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e","Type":"ContainerStarted","Data":"9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5"}
Jul 14 18:25:09.455453 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:09.455416    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb" podStartSLOduration=1.4554023090000001 podStartE2EDuration="1.455402309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:25:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:25:09.453842255 +0000 UTC m=+1827.655397393" watchObservedRunningTime="2026-07-14 18:25:09.455402309 +0000 UTC m=+1827.656957449"
Jul 14 18:25:10.441710 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:10.441679    2556 generic.go:356] "Generic (PLEG): container finished" podID="2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e" containerID="a76e5d9f9b466b9f645b01fe353a15d21042d708a507d4874c1cbe9d92d12be8" exitCode=0
Jul 14 18:25:10.442094 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:10.441752    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb" event={"ID":"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e","Type":"ContainerDied","Data":"a76e5d9f9b466b9f645b01fe353a15d21042d708a507d4874c1cbe9d92d12be8"}
Jul 14 18:25:11.570149 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:11.570127    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:11.707064 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:11.707000    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca\") pod \"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e\" (UID: \"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e\") "
Jul 14 18:25:11.707322 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:11.707300    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e" (UID: "2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:25:11.807502 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:11.807476    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e-maas-api-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:25:12.451801 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:12.451769    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb" event={"ID":"2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e","Type":"ContainerDied","Data":"9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5"}
Jul 14 18:25:12.451920 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:12.451814    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9390ee5b0634f0b1dcdf3aa1b7041fb31ee47604953a0595310386d4f0801bb5"
Jul 14 18:25:12.451920 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:12.451785    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb"
Jul 14 18:25:35.858187 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858150    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:25:35.858604 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858560    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e" containerName="revoke-keys"
Jul 14 18:25:35.858604 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858576    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e" containerName="revoke-keys"
Jul 14 18:25:35.858691 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858619    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:35.858691 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858628    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="177b0101-0f24-4ea0-88b5-a700a3d56797" containerName="revoke-keys"
Jul 14 18:25:35.858751 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.858702    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e" containerName="revoke-keys"
Jul 14 18:25:35.861285 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.861270    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:35.863880 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.863855    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d0b3d618-serving-cert\""
Jul 14 18:25:35.864001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.863904    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:25:35.864573 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.864543    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:25:35.865127 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.865108    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:25:35.871600 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.871563    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:25:35.966099 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.966071    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mznbg\" (UniqueName: \"kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:35.966265 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.966119    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:35.966265 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:35.966202    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.066644 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.066612    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.066797 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.066670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mznbg\" (UniqueName: \"kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.066797 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.066703    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.066797 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:25:36.066763    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-serving-cert: secret "maas-api-e2e-migrate-d0b3d618-serving-cert" not found
Jul 14 18:25:36.066952 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:25:36.066834    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls podName:ec349a14-4874-4b3f-b311-06f29e954101 nodeName:}" failed. No retries permitted until 2026-07-14 18:25:36.566817446 +0000 UTC m=+1854.768372564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls") pod "maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" (UID: "ec349a14-4874-4b3f-b311-06f29e954101") : secret "maas-api-e2e-migrate-d0b3d618-serving-cert" not found
Jul 14 18:25:36.067242 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.067224    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.080443 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.080421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mznbg\" (UniqueName: \"kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.570111 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.570082    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.572486 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.572460    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") pod \"maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.773375 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.773348    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:36.907691 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:36.907402    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:25:36.911318 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:25:36.911289    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec349a14_4874_4b3f_b311_06f29e954101.slice/crio-8d5d0781588ee16c8a8476ad4a4c99746c5668117896e4e1cd482e8883ec8a6d WatchSource:0}: Error finding container 8d5d0781588ee16c8a8476ad4a4c99746c5668117896e4e1cd482e8883ec8a6d: Status 404 returned error can't find the container with id 8d5d0781588ee16c8a8476ad4a4c99746c5668117896e4e1cd482e8883ec8a6d
Jul 14 18:25:37.552645 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:37.552610    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" event={"ID":"ec349a14-4874-4b3f-b311-06f29e954101","Type":"ContainerStarted","Data":"8d5d0781588ee16c8a8476ad4a4c99746c5668117896e4e1cd482e8883ec8a6d"}
Jul 14 18:25:41.572039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:41.572000    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" event={"ID":"ec349a14-4874-4b3f-b311-06f29e954101","Type":"ContainerStarted","Data":"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"}
Jul 14 18:25:41.572435 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:41.572083    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:25:41.612532 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:41.612483    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" podStartSLOduration=2.451002658 podStartE2EDuration="6.612466912s" totalImagesPullingTime="4.161464254s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:25:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:25:41.61135305 +0000 UTC m=+1859.812908190" watchObservedRunningTime="2026-07-14 18:25:41.612466912 +0000 UTC m=+1859.814022054"
Jul 14 18:25:47.581974 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:25:47.581945    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:26:45.820882 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:45.820851    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:26:45.821361 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:45.821120    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" podUID="ec349a14-4874-4b3f-b311-06f29e954101" containerName="maas-api" containerID="cri-o://ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2" gracePeriod=30
Jul 14 18:26:46.064929 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.064901    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:26:46.162338 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.162315    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") pod \"ec349a14-4874-4b3f-b311-06f29e954101\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") "
Jul 14 18:26:46.162484 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.162365    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca\") pod \"ec349a14-4874-4b3f-b311-06f29e954101\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") "
Jul 14 18:26:46.162484 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.162406    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mznbg\" (UniqueName: \"kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg\") pod \"ec349a14-4874-4b3f-b311-06f29e954101\" (UID: \"ec349a14-4874-4b3f-b311-06f29e954101\") "
Jul 14 18:26:46.162805 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.162782    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ec349a14-4874-4b3f-b311-06f29e954101" (UID: "ec349a14-4874-4b3f-b311-06f29e954101"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:26:46.164615 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.164578    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ec349a14-4874-4b3f-b311-06f29e954101" (UID: "ec349a14-4874-4b3f-b311-06f29e954101"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:26:46.164695 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.164630    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg" (OuterVolumeSpecName: "kube-api-access-mznbg") pod "ec349a14-4874-4b3f-b311-06f29e954101" (UID: "ec349a14-4874-4b3f-b311-06f29e954101"). InnerVolumeSpecName "kube-api-access-mznbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:26:46.263741 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.263688    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ec349a14-4874-4b3f-b311-06f29e954101-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:26:46.263741 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.263710    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ec349a14-4874-4b3f-b311-06f29e954101-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:26:46.263741 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.263720    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mznbg\" (UniqueName: \"kubernetes.io/projected/ec349a14-4874-4b3f-b311-06f29e954101-kube-api-access-mznbg\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:26:46.836152 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.836113    2556 generic.go:356] "Generic (PLEG): container finished" podID="ec349a14-4874-4b3f-b311-06f29e954101" containerID="ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2" exitCode=0
Jul 14 18:26:46.836574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.836159    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"
Jul 14 18:26:46.836574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.836168    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" event={"ID":"ec349a14-4874-4b3f-b311-06f29e954101","Type":"ContainerDied","Data":"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"}
Jul 14 18:26:46.836574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.836225    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh" event={"ID":"ec349a14-4874-4b3f-b311-06f29e954101","Type":"ContainerDied","Data":"8d5d0781588ee16c8a8476ad4a4c99746c5668117896e4e1cd482e8883ec8a6d"}
Jul 14 18:26:46.836574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.836248    2556 scope.go:117] "RemoveContainer" containerID="ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"
Jul 14 18:26:46.844386 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.844365    2556 scope.go:117] "RemoveContainer" containerID="ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"
Jul 14 18:26:46.844657 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:26:46.844632    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2\": container with ID starting with ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2 not found: ID does not exist" containerID="ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"
Jul 14 18:26:46.844721 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.844668    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2"} err="failed to get container status \"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2\": rpc error: code = NotFound desc = could not find container \"ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2\": container with ID starting with ab98d32d993481cb3d5d6ac5c508e9e920321d2873faaee308e2050e194134f2 not found: ID does not exist"
Jul 14 18:26:46.853888 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.853865    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:26:46.858162 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:46.858136    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0b3d618-7dddd5bc5d-v5cwh"]
Jul 14 18:26:48.319411 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:48.319379    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec349a14-4874-4b3f-b311-06f29e954101" path="/var/lib/kubelet/pods/ec349a14-4874-4b3f-b311-06f29e954101/volumes"
Jul 14 18:26:54.208763 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.208733    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:26:54.209190 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.209119    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec349a14-4874-4b3f-b311-06f29e954101" containerName="maas-api"
Jul 14 18:26:54.209190 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.209136    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec349a14-4874-4b3f-b311-06f29e954101" containerName="maas-api"
Jul 14 18:26:54.209190 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.209181    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec349a14-4874-4b3f-b311-06f29e954101" containerName="maas-api"
Jul 14 18:26:54.213901 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.213875    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.216209 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.216183    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-296e74aa-serving-cert\""
Jul 14 18:26:54.216345 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.216206    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:26:54.216712 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.216691    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:26:54.217356 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.217336    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:26:54.222155 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.222127    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:26:54.327559 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.327532    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.327559 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.327558    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49x8\" (UniqueName: \"kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.327729 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.327647    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.428182 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.428161    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.428294 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.428190    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49x8\" (UniqueName: \"kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.428294 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.428257    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.428904 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.428881    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.430785 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.430762    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.439336 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.439313    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49x8\" (UniqueName: \"kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8\") pod \"maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.525562 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.525514    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:54.652432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.652393    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:26:54.657062 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:26:54.657028    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e788376_8140_4628_9684_f4b941b6ca54.slice/crio-4de88997b1678dcab35f04d99dc748d5de82c3896b5217fcb0ab001f41dc839e WatchSource:0}: Error finding container 4de88997b1678dcab35f04d99dc748d5de82c3896b5217fcb0ab001f41dc839e: Status 404 returned error can't find the container with id 4de88997b1678dcab35f04d99dc748d5de82c3896b5217fcb0ab001f41dc839e
Jul 14 18:26:54.871932 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:54.871901    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" event={"ID":"6e788376-8140-4628-9684-f4b941b6ca54","Type":"ContainerStarted","Data":"4de88997b1678dcab35f04d99dc748d5de82c3896b5217fcb0ab001f41dc839e"}
Jul 14 18:26:55.877009 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:55.876969    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" event={"ID":"6e788376-8140-4628-9684-f4b941b6ca54","Type":"ContainerStarted","Data":"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"}
Jul 14 18:26:55.877379 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:55.877044    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:26:55.898261 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:26:55.898212    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" podStartSLOduration=1.3824971449999999 podStartE2EDuration="1.898191424s" totalImagesPullingTime="515.694279ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:26:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:26:55.897072924 +0000 UTC m=+1934.098628063" watchObservedRunningTime="2026-07-14 18:26:55.898191424 +0000 UTC m=+1934.099746564"
Jul 14 18:27:01.886898 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:01.886871    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:27:10.005036 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.005002    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:27:10.005443 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.005226    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" podUID="6e788376-8140-4628-9684-f4b941b6ca54" containerName="maas-api" containerID="cri-o://e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599" gracePeriod=30
Jul 14 18:27:10.247983 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.247953    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:27:10.347144 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.347117    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c49x8\" (UniqueName: \"kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8\") pod \"6e788376-8140-4628-9684-f4b941b6ca54\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") "
Jul 14 18:27:10.347276 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.347186    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca\") pod \"6e788376-8140-4628-9684-f4b941b6ca54\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") "
Jul 14 18:27:10.347276 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.347218    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls\") pod \"6e788376-8140-4628-9684-f4b941b6ca54\" (UID: \"6e788376-8140-4628-9684-f4b941b6ca54\") "
Jul 14 18:27:10.347536 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.347512    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6e788376-8140-4628-9684-f4b941b6ca54" (UID: "6e788376-8140-4628-9684-f4b941b6ca54"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:27:10.349375 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.349352    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8" (OuterVolumeSpecName: "kube-api-access-c49x8") pod "6e788376-8140-4628-9684-f4b941b6ca54" (UID: "6e788376-8140-4628-9684-f4b941b6ca54"). InnerVolumeSpecName "kube-api-access-c49x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:27:10.349453 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.349370    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6e788376-8140-4628-9684-f4b941b6ca54" (UID: "6e788376-8140-4628-9684-f4b941b6ca54"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:27:10.448095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.448073    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6e788376-8140-4628-9684-f4b941b6ca54-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:27:10.448095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.448096    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c49x8\" (UniqueName: \"kubernetes.io/projected/6e788376-8140-4628-9684-f4b941b6ca54-kube-api-access-c49x8\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:27:10.448280 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.448105    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e788376-8140-4628-9684-f4b941b6ca54-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:27:10.938680 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.938645    2556 generic.go:356] "Generic (PLEG): container finished" podID="6e788376-8140-4628-9684-f4b941b6ca54" containerID="e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599" exitCode=0
Jul 14 18:27:10.938834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.938712    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"
Jul 14 18:27:10.938834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.938722    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" event={"ID":"6e788376-8140-4628-9684-f4b941b6ca54","Type":"ContainerDied","Data":"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"}
Jul 14 18:27:10.938834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.938797    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42" event={"ID":"6e788376-8140-4628-9684-f4b941b6ca54","Type":"ContainerDied","Data":"4de88997b1678dcab35f04d99dc748d5de82c3896b5217fcb0ab001f41dc839e"}
Jul 14 18:27:10.938834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.938818    2556 scope.go:117] "RemoveContainer" containerID="e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"
Jul 14 18:27:10.946959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.946942    2556 scope.go:117] "RemoveContainer" containerID="e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"
Jul 14 18:27:10.947194 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:27:10.947177    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599\": container with ID starting with e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599 not found: ID does not exist" containerID="e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"
Jul 14 18:27:10.947252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.947202    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599"} err="failed to get container status \"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599\": rpc error: code = NotFound desc = could not find container \"e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599\": container with ID starting with e4fe2d99ea251ab5a279542ff9d420ff9586ae5664772cb18d0f5b5e509a0599 not found: ID does not exist"
Jul 14 18:27:10.958033 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.958013    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:27:10.964631 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:10.964605    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-296e74aa-85c8b9c778-xjj42"]
Jul 14 18:27:12.319325 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:12.319295    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e788376-8140-4628-9684-f4b941b6ca54" path="/var/lib/kubelet/pods/6e788376-8140-4628-9684-f4b941b6ca54/volumes"
Jul 14 18:27:26.869623 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.869577    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:27:26.870026 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.869899    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e788376-8140-4628-9684-f4b941b6ca54" containerName="maas-api"
Jul 14 18:27:26.870026 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.869910    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e788376-8140-4628-9684-f4b941b6ca54" containerName="maas-api"
Jul 14 18:27:26.870026 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.869972    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e788376-8140-4628-9684-f4b941b6ca54" containerName="maas-api"
Jul 14 18:27:26.873007 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.872988    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:26.876884 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.876763    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:27:26.877012 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.876964    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:27:26.877336 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.877320    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-97e17a87-serving-cert\""
Jul 14 18:27:26.878874 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.878853    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:27:26.887053 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.887033    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:27:26.968739 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.968717    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:26.968834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.968763    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htm5t\" (UniqueName: \"kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:26.968834 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:26.968803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.069402 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.069381    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.069517 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.069426    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htm5t\" (UniqueName: \"kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.069517 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.069449    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.070110 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.070091    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.071643 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.071625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.078957 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.078937    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htm5t\" (UniqueName: \"kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t\") pod \"maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.182727 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.182659    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:27.317733 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:27.317711    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:27:27.319546 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:27:27.319516    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e1a1fd_ff75_4821_9dba_2cfa951fc056.slice/crio-4cb11f7de32b1be90c54d10836b9266ec6cb4e887c415fa53a9c17d01ccdc574 WatchSource:0}: Error finding container 4cb11f7de32b1be90c54d10836b9266ec6cb4e887c415fa53a9c17d01ccdc574: Status 404 returned error can't find the container with id 4cb11f7de32b1be90c54d10836b9266ec6cb4e887c415fa53a9c17d01ccdc574
Jul 14 18:27:28.010251 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:28.010114    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" event={"ID":"d7e1a1fd-ff75-4821-9dba-2cfa951fc056","Type":"ContainerStarted","Data":"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"}
Jul 14 18:27:28.010251 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:28.010238    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:27:28.010746 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:28.010264    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" event={"ID":"d7e1a1fd-ff75-4821-9dba-2cfa951fc056","Type":"ContainerStarted","Data":"4cb11f7de32b1be90c54d10836b9266ec6cb4e887c415fa53a9c17d01ccdc574"}
Jul 14 18:27:28.047047 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:28.046996    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" podStartSLOduration=1.668568383 podStartE2EDuration="2.046977418s" totalImagesPullingTime="378.409035ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:27:28.046101215 +0000 UTC m=+1966.247656355" watchObservedRunningTime="2026-07-14 18:27:28.046977418 +0000 UTC m=+1966.248532559"
Jul 14 18:27:34.020878 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:27:34.020798    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:28:41.717363 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:41.717331    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:28:41.717806 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:41.717555    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" podUID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" containerName="maas-api" containerID="cri-o://c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9" gracePeriod=30
Jul 14 18:28:41.950744 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:41.950718    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:28:42.044094 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.044009    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls\") pod \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") "
Jul 14 18:28:42.044249 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.044132    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca\") pod \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") "
Jul 14 18:28:42.044249 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.044182    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htm5t\" (UniqueName: \"kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t\") pod \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\" (UID: \"d7e1a1fd-ff75-4821-9dba-2cfa951fc056\") "
Jul 14 18:28:42.044523 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.044495    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d7e1a1fd-ff75-4821-9dba-2cfa951fc056" (UID: "d7e1a1fd-ff75-4821-9dba-2cfa951fc056"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:28:42.046305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.046279    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t" (OuterVolumeSpecName: "kube-api-access-htm5t") pod "d7e1a1fd-ff75-4821-9dba-2cfa951fc056" (UID: "d7e1a1fd-ff75-4821-9dba-2cfa951fc056"). InnerVolumeSpecName "kube-api-access-htm5t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:28:42.046484 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.046452    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d7e1a1fd-ff75-4821-9dba-2cfa951fc056" (UID: "d7e1a1fd-ff75-4821-9dba-2cfa951fc056"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:28:42.144934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.144905    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:28:42.144934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.144931    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htm5t\" (UniqueName: \"kubernetes.io/projected/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-kube-api-access-htm5t\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:28:42.144934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.144941    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7e1a1fd-ff75-4821-9dba-2cfa951fc056-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:28:42.329013 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.328985    2556 generic.go:356] "Generic (PLEG): container finished" podID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" containerID="c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9" exitCode=0
Jul 14 18:28:42.329134 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.329037    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"
Jul 14 18:28:42.329134 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.329046    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" event={"ID":"d7e1a1fd-ff75-4821-9dba-2cfa951fc056","Type":"ContainerDied","Data":"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"}
Jul 14 18:28:42.329134 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.329083    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl" event={"ID":"d7e1a1fd-ff75-4821-9dba-2cfa951fc056","Type":"ContainerDied","Data":"4cb11f7de32b1be90c54d10836b9266ec6cb4e887c415fa53a9c17d01ccdc574"}
Jul 14 18:28:42.329134 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.329104    2556 scope.go:117] "RemoveContainer" containerID="c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"
Jul 14 18:28:42.337032 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.337017    2556 scope.go:117] "RemoveContainer" containerID="c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"
Jul 14 18:28:42.337273 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:28:42.337250    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9\": container with ID starting with c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9 not found: ID does not exist" containerID="c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"
Jul 14 18:28:42.337332 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.337280    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9"} err="failed to get container status \"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9\": rpc error: code = NotFound desc = could not find container \"c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9\": container with ID starting with c2cf5b42826a78526a952d5781ec98cae54c631ccd7da7174bbd4f4d4942dde9 not found: ID does not exist"
Jul 14 18:28:42.345411 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.345390    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:28:42.352053 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:42.352036    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-97e17a87-6f8b7659dc-wqzsl"]
Jul 14 18:28:44.319111 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:28:44.319078    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" path="/var/lib/kubelet/pods/d7e1a1fd-ff75-4821-9dba-2cfa951fc056/volumes"
Jul 14 18:29:42.287147 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:29:42.287118    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:29:42.292952 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:29:42.292931    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:29:42.294123 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:29:42.294093    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:29:42.298998 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:29:42.298982    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:30:38.437637 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.437542    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:30:38.438175 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.437988    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" containerName="maas-api"
Jul 14 18:30:38.438175 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.438007    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" containerName="maas-api"
Jul 14 18:30:38.438175 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.438088    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e1a1fd-ff75-4821-9dba-2cfa951fc056" containerName="maas-api"
Jul 14 18:30:38.440999 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.440977    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.443434 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.443413    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-c1e47003-serving-cert\""
Jul 14 18:30:38.443548 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.443471    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:30:38.444233 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.444213    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:30:38.444332 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.444249    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:30:38.451470 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.451412    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:30:38.615715 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.615684    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.615862 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.615752    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhrpt\" (UniqueName: \"kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.615927 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.615867    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.717010 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.716950    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhrpt\" (UniqueName: \"kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.717108 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.717012    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.717108 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.717041    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.717702 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.717685    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.719467 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.719451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.726209 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.726183    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhrpt\" (UniqueName: \"kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt\") pod \"maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.755047 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.755027    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:38.882812 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.882778    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:30:38.885355 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:30:38.885331    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75dbf8e8_f31e_4dbd_9c70_4d6d8850e2b6.slice/crio-62d1b6743af7db8dbab8c4dfd9dfa3d68599862ddd041309ee27a4bcc110c333 WatchSource:0}: Error finding container 62d1b6743af7db8dbab8c4dfd9dfa3d68599862ddd041309ee27a4bcc110c333: Status 404 returned error can't find the container with id 62d1b6743af7db8dbab8c4dfd9dfa3d68599862ddd041309ee27a4bcc110c333
Jul 14 18:30:38.887022 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:38.887004    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 18:30:39.817061 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:39.816976    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" event={"ID":"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6","Type":"ContainerStarted","Data":"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"}
Jul 14 18:30:39.817061 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:39.817031    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" event={"ID":"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6","Type":"ContainerStarted","Data":"62d1b6743af7db8dbab8c4dfd9dfa3d68599862ddd041309ee27a4bcc110c333"}
Jul 14 18:30:39.817468 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:39.817123    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:30:39.836159 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:39.836111    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" podStartSLOduration=1.273766712 podStartE2EDuration="1.836096283s" totalImagesPullingTime="562.329571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:30:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:30:39.835682615 +0000 UTC m=+2158.037237755" watchObservedRunningTime="2026-07-14 18:30:39.836096283 +0000 UTC m=+2158.037651423"
Jul 14 18:30:42.367967 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:42.367941    2556 scope.go:117] "RemoveContainer" containerID="678e779264aff6203c073b1b2435b50020cec117921636da45d8510616b3f1c2"
Jul 14 18:30:45.828102 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:30:45.828072    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:31:00.902142 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:00.902095    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:31:00.902516 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:00.902399    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" podUID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" containerName="maas-api" containerID="cri-o://bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865" gracePeriod=30
Jul 14 18:31:01.148952 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.148928    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:31:01.181611 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.181535    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca\") pod \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") "
Jul 14 18:31:01.181730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.181628    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls\") pod \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") "
Jul 14 18:31:01.181730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.181656    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhrpt\" (UniqueName: \"kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt\") pod \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\" (UID: \"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6\") "
Jul 14 18:31:01.181973 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.181945    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" (UID: "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:31:01.183711 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.183685    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt" (OuterVolumeSpecName: "kube-api-access-qhrpt") pod "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" (UID: "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6"). InnerVolumeSpecName "kube-api-access-qhrpt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:31:01.183807 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.183778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" (UID: "75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:31:01.282608 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.282556    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:31:01.282608 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.282606    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhrpt\" (UniqueName: \"kubernetes.io/projected/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-kube-api-access-qhrpt\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:31:01.282746 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.282623    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:31:01.913807 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.913771    2556 generic.go:356] "Generic (PLEG): container finished" podID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" containerID="bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865" exitCode=0
Jul 14 18:31:01.914188 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.913808    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"
Jul 14 18:31:01.914188 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.913848    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" event={"ID":"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6","Type":"ContainerDied","Data":"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"}
Jul 14 18:31:01.914188 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.913902    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj" event={"ID":"75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6","Type":"ContainerDied","Data":"62d1b6743af7db8dbab8c4dfd9dfa3d68599862ddd041309ee27a4bcc110c333"}
Jul 14 18:31:01.914188 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.913929    2556 scope.go:117] "RemoveContainer" containerID="bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"
Jul 14 18:31:01.922546 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.922527    2556 scope.go:117] "RemoveContainer" containerID="bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"
Jul 14 18:31:01.922867 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:31:01.922847    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865\": container with ID starting with bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865 not found: ID does not exist" containerID="bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"
Jul 14 18:31:01.922932 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.922878    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865"} err="failed to get container status \"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865\": rpc error: code = NotFound desc = could not find container \"bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865\": container with ID starting with bdfbe540b815a31d0d62e1856193eb17cd7ca1d4abdddc0d198ee83b137ce865 not found: ID does not exist"
Jul 14 18:31:01.935699 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.935676    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:31:01.941538 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:01.941518    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-c1e47003-64d87f5f7c-xw9cj"]
Jul 14 18:31:02.318396 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:02.318322    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" path="/var/lib/kubelet/pods/75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6/volumes"
Jul 14 18:31:40.874604 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.874553    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:31:40.875030 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.874883    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" containerName="maas-api"
Jul 14 18:31:40.875030 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.874899    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" containerName="maas-api"
Jul 14 18:31:40.875030 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.874962    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="75dbf8e8-f31e-4dbd-9c70-4d6d8850e2b6" containerName="maas-api"
Jul 14 18:31:40.878016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.877999    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:40.882695 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.882653    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-31aded09-serving-cert\""
Jul 14 18:31:40.882695 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.882666    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:31:40.883381 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.883359    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:31:40.883499 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.883368    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:31:40.891688 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.891665    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:31:40.983505 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.983482    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:40.983650 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.983525    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhvtd\" (UniqueName: \"kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:40.983650 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:40.983576    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.084534 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.084512    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.084667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.084550    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhvtd\" (UniqueName: \"kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.084667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.084576    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.084779 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:31:41.084680    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-serving-cert: secret "maas-api-e2e-mt-31aded09-serving-cert" not found
Jul 14 18:31:41.084779 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:31:41.084763    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls podName:49d03fd0-fe2c-43e3-b781-238cf21a86a3 nodeName:}" failed. No retries permitted until 2026-07-14 18:31:41.584741368 +0000 UTC m=+2219.786296502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls") pod "maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" (UID: "49d03fd0-fe2c-43e3-b781-238cf21a86a3") : secret "maas-api-e2e-mt-31aded09-serving-cert" not found
Jul 14 18:31:41.085150 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.085132    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.095705 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.095684    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhvtd\" (UniqueName: \"kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.589478 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.589443    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.591918 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.591896    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") pod \"maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.788724 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.788693    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:41.919384 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:41.919353    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:31:41.921320 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:31:41.921287    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d03fd0_fe2c_43e3_b781_238cf21a86a3.slice/crio-0d32f7382367165af89dfbbd0413e781219a800779f3d5034ae9bfceb6effe26 WatchSource:0}: Error finding container 0d32f7382367165af89dfbbd0413e781219a800779f3d5034ae9bfceb6effe26: Status 404 returned error can't find the container with id 0d32f7382367165af89dfbbd0413e781219a800779f3d5034ae9bfceb6effe26
Jul 14 18:31:42.099205 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:42.099173    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" event={"ID":"49d03fd0-fe2c-43e3-b781-238cf21a86a3","Type":"ContainerStarted","Data":"0d32f7382367165af89dfbbd0413e781219a800779f3d5034ae9bfceb6effe26"}
Jul 14 18:31:43.104902 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:43.104871    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" event={"ID":"49d03fd0-fe2c-43e3-b781-238cf21a86a3","Type":"ContainerStarted","Data":"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"}
Jul 14 18:31:43.105262 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:43.104925    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:43.139600 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:43.139542    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" podStartSLOduration=2.787125931 podStartE2EDuration="3.139530263s" totalImagesPullingTime="352.404332ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:31:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:31:43.139180439 +0000 UTC m=+2221.340735580" watchObservedRunningTime="2026-07-14 18:31:43.139530263 +0000 UTC m=+2221.341085443"
Jul 14 18:31:49.114320 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:49.114290    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:31:54.265054 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.265025    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:31:54.268323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.268301    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.271549 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.271524    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-31aded09\"/\"default-dockercfg-8sgkx\""
Jul 14 18:31:54.271654 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.271524    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-31aded09\"/\"openshift-service-ca.crt\""
Jul 14 18:31:54.271654 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.271524    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-31aded09\"/\"e2e-lifecycle-model-31aded09-kserve-self-signed-certs\""
Jul 14 18:31:54.271763 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.271555    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-31aded09\"/\"kube-root-ca.crt\""
Jul 14 18:31:54.285918 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.285896    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:31:54.381324 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381300    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.381424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381339    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzxhn\" (UniqueName: \"kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.381424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381363    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.381493 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381434    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.381493 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381467    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.381493 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.381488    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482222 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482201    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482342 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482248    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzxhn\" (UniqueName: \"kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482342 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482275    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482342 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482311    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482342 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482338    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482541 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482678 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482659    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482732 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482715    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.482775 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.482729    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.484613 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.484564    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.484902 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.484885    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.493341 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.493318    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzxhn\" (UniqueName: \"kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn\") pod \"e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") " pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.577049 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.576997    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:31:54.704733 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:54.704684    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:31:54.707167 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:31:54.707141    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a4c5dd3_e7c0_44ce_b34f_87c53f398640.slice/crio-032e2f2eac826f7fadcf6f815cb73f7f673668a8152dc96ceca2d091fa2c5bb9 WatchSource:0}: Error finding container 032e2f2eac826f7fadcf6f815cb73f7f673668a8152dc96ceca2d091fa2c5bb9: Status 404 returned error can't find the container with id 032e2f2eac826f7fadcf6f815cb73f7f673668a8152dc96ceca2d091fa2c5bb9
Jul 14 18:31:55.157667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:55.157622    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" event={"ID":"1a4c5dd3-e7c0-44ce-b34f-87c53f398640","Type":"ContainerStarted","Data":"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"}
Jul 14 18:31:55.157667 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:55.157675    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" event={"ID":"1a4c5dd3-e7c0-44ce-b34f-87c53f398640","Type":"ContainerStarted","Data":"032e2f2eac826f7fadcf6f815cb73f7f673668a8152dc96ceca2d091fa2c5bb9"}
Jul 14 18:31:55.178266 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:31:55.178224    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" podStartSLOduration=1.178210072 podStartE2EDuration="1.178210072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:31:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:31:55.177274224 +0000 UTC m=+2233.378829364" watchObservedRunningTime="2026-07-14 18:31:55.178210072 +0000 UTC m=+2233.379765254"
Jul 14 18:32:04.577770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:04.577691    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:32:04.577770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:04.577737    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:32:04.589603 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:04.589565    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:32:05.209637 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:05.209611    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:32:39.957554 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:39.957513    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:32:39.958105 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:39.957732    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" podUID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" containerName="maas-api" containerID="cri-o://044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d" gracePeriod=30
Jul 14 18:32:40.194859 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.194831    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:32:40.331300 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.331238    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") pod \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") "
Jul 14 18:32:40.331435 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.331308    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca\") pod \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") "
Jul 14 18:32:40.331435 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.331362    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhvtd\" (UniqueName: \"kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd\") pod \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\" (UID: \"49d03fd0-fe2c-43e3-b781-238cf21a86a3\") "
Jul 14 18:32:40.331619 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.331574    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49d03fd0-fe2c-43e3-b781-238cf21a86a3" (UID: "49d03fd0-fe2c-43e3-b781-238cf21a86a3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:32:40.333392 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.333371    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd" (OuterVolumeSpecName: "kube-api-access-bhvtd") pod "49d03fd0-fe2c-43e3-b781-238cf21a86a3" (UID: "49d03fd0-fe2c-43e3-b781-238cf21a86a3"). InnerVolumeSpecName "kube-api-access-bhvtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:32:40.333495 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.333451    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49d03fd0-fe2c-43e3-b781-238cf21a86a3" (UID: "49d03fd0-fe2c-43e3-b781-238cf21a86a3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:32:40.342964 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.342939    2556 generic.go:356] "Generic (PLEG): container finished" podID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" containerID="044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d" exitCode=0
Jul 14 18:32:40.343051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.342984    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"
Jul 14 18:32:40.343051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.343020    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" event={"ID":"49d03fd0-fe2c-43e3-b781-238cf21a86a3","Type":"ContainerDied","Data":"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"}
Jul 14 18:32:40.343163 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.343084    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r" event={"ID":"49d03fd0-fe2c-43e3-b781-238cf21a86a3","Type":"ContainerDied","Data":"0d32f7382367165af89dfbbd0413e781219a800779f3d5034ae9bfceb6effe26"}
Jul 14 18:32:40.343163 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.343118    2556 scope.go:117] "RemoveContainer" containerID="044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"
Jul 14 18:32:40.352431 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.352414    2556 scope.go:117] "RemoveContainer" containerID="044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"
Jul 14 18:32:40.352757 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:32:40.352734    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d\": container with ID starting with 044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d not found: ID does not exist" containerID="044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"
Jul 14 18:32:40.352817 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.352765    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d"} err="failed to get container status \"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d\": rpc error: code = NotFound desc = could not find container \"044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d\": container with ID starting with 044deb1743d4fcdfa0b19bc4fbf6f05450dbe33d05def7189655b3373a2ac64d not found: ID does not exist"
Jul 14 18:32:40.366395 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.366375    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:32:40.372437 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.372418    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-31aded09-7644f7cf45-c8j2r"]
Jul 14 18:32:40.432984 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.432957    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhvtd\" (UniqueName: \"kubernetes.io/projected/49d03fd0-fe2c-43e3-b781-238cf21a86a3-kube-api-access-bhvtd\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:32:40.433091 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.432989    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d03fd0-fe2c-43e3-b781-238cf21a86a3-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:32:40.433091 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:40.433005    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d03fd0-fe2c-43e3-b781-238cf21a86a3-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:32:42.323860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:42.323827    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" path="/var/lib/kubelet/pods/49d03fd0-fe2c-43e3-b781-238cf21a86a3/volumes"
Jul 14 18:32:49.334418 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:32:49.334379    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:33:04.358933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.358874    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" podUID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" containerName="main" containerID="cri-o://2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e" gracePeriod=45
Jul 14 18:33:04.600872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.600850    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:33:04.712755 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712687    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712755 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712714    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712755 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712747    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712831    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712879    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzxhn\" (UniqueName: \"kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712905    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache\") pod \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\" (UID: \"1a4c5dd3-e7c0-44ce-b34f-87c53f398640\") "
Jul 14 18:33:04.712986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.712978    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:33:04.713201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.713018    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home" (OuterVolumeSpecName: "home") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:33:04.713201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.713162    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache" (OuterVolumeSpecName: "model-cache") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:33:04.713201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.713182    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:04.713201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.713196    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:04.714952 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.714925    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm" (OuterVolumeSpecName: "dshm") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:33:04.715049 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.715032    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn" (OuterVolumeSpecName: "kube-api-access-vzxhn") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "kube-api-access-vzxhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:33:04.715095 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.715044    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1a4c5dd3-e7c0-44ce-b34f-87c53f398640" (UID: "1a4c5dd3-e7c0-44ce-b34f-87c53f398640"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:33:04.814104 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.814082    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:04.814104 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.814101    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:04.814229 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.814110    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzxhn\" (UniqueName: \"kubernetes.io/projected/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-kube-api-access-vzxhn\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:04.814229 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:04.814118    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1a4c5dd3-e7c0-44ce-b34f-87c53f398640-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:33:05.450461 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.450432    2556 generic.go:356] "Generic (PLEG): container finished" podID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" containerID="2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e" exitCode=0
Jul 14 18:33:05.450899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.450468    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"
Jul 14 18:33:05.450899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.450524    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" event={"ID":"1a4c5dd3-e7c0-44ce-b34f-87c53f398640","Type":"ContainerDied","Data":"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"}
Jul 14 18:33:05.450899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.450568    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq" event={"ID":"1a4c5dd3-e7c0-44ce-b34f-87c53f398640","Type":"ContainerDied","Data":"032e2f2eac826f7fadcf6f815cb73f7f673668a8152dc96ceca2d091fa2c5bb9"}
Jul 14 18:33:05.450899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.450618    2556 scope.go:117] "RemoveContainer" containerID="2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"
Jul 14 18:33:05.458987 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.458970    2556 scope.go:117] "RemoveContainer" containerID="2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"
Jul 14 18:33:05.459213 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:33:05.459197    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e\": container with ID starting with 2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e not found: ID does not exist" containerID="2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"
Jul 14 18:33:05.459258 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.459221    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e"} err="failed to get container status \"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e\": rpc error: code = NotFound desc = could not find container \"2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e\": container with ID starting with 2e7136cf2ab8ed9852e1108791fa84cfdf444d8dc69d5ea40752e5bb618b854e not found: ID does not exist"
Jul 14 18:33:05.477521 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.477497    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:33:05.483380 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:05.483361    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-31aded09/e2e-lifecycle-model-31aded09-kserve-74ff78f99-gqdvq"]
Jul 14 18:33:06.319032 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:33:06.318992    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" path="/var/lib/kubelet/pods/1a4c5dd3-e7c0-44ce-b34f-87c53f398640/volumes"
Jul 14 18:34:42.310934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:42.310907    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:34:42.316925 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:42.316904    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:34:42.317976 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:42.317955    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:34:42.323712 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:42.323695    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:34:48.042670 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.042641    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.042926    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" containerName="main"
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.042936    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" containerName="main"
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.042944    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" containerName="maas-api"
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.042949    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" containerName="maas-api"
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.043002    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="49d03fd0-fe2c-43e3-b781-238cf21a86a3" containerName="maas-api"
Jul 14 18:34:48.043016 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.043010    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a4c5dd3-e7c0-44ce-b34f-87c53f398640" containerName="main"
Jul 14 18:34:48.045717 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.045692    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.048859 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.048835    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-4785d6-serving-cert\""
Jul 14 18:34:48.049007 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.048990    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:34:48.049426 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.049413    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:34:48.055035 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.055017    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:34:48.064411 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.064388    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:34:48.130991 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.130968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.131093 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.131049    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.131133 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.131113    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9lr5\" (UniqueName: \"kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.231668 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.231648    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.231764 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.231700    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9lr5\" (UniqueName: \"kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.231828 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.231806    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.232366 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.232345    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.234308 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.234281    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.242913 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.242888    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9lr5\" (UniqueName: \"kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5\") pod \"maas-api-e2e-inf-a-4785d6-c95876c98-754vp\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.357462 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.357443    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:48.491702 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.491674    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:34:48.494562 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:34:48.494534    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfed9c65f_8f86_4df7_a560_60eec2e80ebf.slice/crio-9d5ddba111faf9dff143dae75dd4e580cb70e6a9c28566857a3fe1fc5f507aa4 WatchSource:0}: Error finding container 9d5ddba111faf9dff143dae75dd4e580cb70e6a9c28566857a3fe1fc5f507aa4: Status 404 returned error can't find the container with id 9d5ddba111faf9dff143dae75dd4e580cb70e6a9c28566857a3fe1fc5f507aa4
Jul 14 18:34:48.882830 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:48.882801    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" event={"ID":"fed9c65f-8f86-4df7-a560-60eec2e80ebf","Type":"ContainerStarted","Data":"9d5ddba111faf9dff143dae75dd4e580cb70e6a9c28566857a3fe1fc5f507aa4"}
Jul 14 18:34:51.895918 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:51.895878    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" event={"ID":"fed9c65f-8f86-4df7-a560-60eec2e80ebf","Type":"ContainerStarted","Data":"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"}
Jul 14 18:34:51.896305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:51.895995    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:51.917652 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:51.917607    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" podStartSLOduration=1.139934097 podStartE2EDuration="3.917569228s" totalImagesPullingTime="2.777635131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:34:51.914683387 +0000 UTC m=+2410.116238527" watchObservedRunningTime="2026-07-14 18:34:51.917569228 +0000 UTC m=+2410.119124368"
Jul 14 18:34:56.089206 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.089174    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:34:56.092504 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.092486    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.095159 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.095141    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-55e12d-serving-cert\""
Jul 14 18:34:56.113758 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.113735    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:34:56.199388 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.199361    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.199506 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.199421    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.199506 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.199465    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fllvx\" (UniqueName: \"kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.300437 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.300411    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.300569 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.300549    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.300639 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.300626    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fllvx\" (UniqueName: \"kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.301151 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.301128    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.302933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.302906    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.309999 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.309975    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fllvx\" (UniqueName: \"kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx\") pod \"maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.403304 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.403286    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:56.552934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.552906    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:34:56.554934 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:34:56.554904    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176262ec_a4f8_4b1b_8578_16079a61c4c0.slice/crio-ed0cb1d3d5c1cec62d150e383846dad749030dfac26d845a54ebf7355d4b75c9 WatchSource:0}: Error finding container ed0cb1d3d5c1cec62d150e383846dad749030dfac26d845a54ebf7355d4b75c9: Status 404 returned error can't find the container with id ed0cb1d3d5c1cec62d150e383846dad749030dfac26d845a54ebf7355d4b75c9
Jul 14 18:34:56.916938 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:56.916911    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" event={"ID":"176262ec-a4f8-4b1b-8578-16079a61c4c0","Type":"ContainerStarted","Data":"ed0cb1d3d5c1cec62d150e383846dad749030dfac26d845a54ebf7355d4b75c9"}
Jul 14 18:34:57.904203 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:57.904174    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:34:58.927182 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:58.927110    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" event={"ID":"176262ec-a4f8-4b1b-8578-16079a61c4c0","Type":"ContainerStarted","Data":"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"}
Jul 14 18:34:58.927495 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:58.927194    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:34:58.948452 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:34:58.948401    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" podStartSLOduration=0.846931616 podStartE2EDuration="2.948383544s" totalImagesPullingTime="2.101451928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:34:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:34:58.947950177 +0000 UTC m=+2417.149505313" watchObservedRunningTime="2026-07-14 18:34:58.948383544 +0000 UTC m=+2417.149938684"
Jul 14 18:35:04.938819 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:04.938749    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:35:28.276058 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.276030    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:35:28.279313 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.279298    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.282513 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.282488    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4785d6\"/\"default-dockercfg-ppf7d\""
Jul 14 18:35:28.282513 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.282513    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4785d6\"/\"kube-root-ca.crt\""
Jul 14 18:35:28.282748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.282535    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4785d6\"/\"test-model-4785d6-kserve-self-signed-certs\""
Jul 14 18:35:28.282748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.282618    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4785d6\"/\"openshift-service-ca.crt\""
Jul 14 18:35:28.288734 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.288716    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:35:28.343904 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.343877    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.344032 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.343916    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.344032 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.343943    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.344103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.344050    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwfp5\" (UniqueName: \"kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.344135 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.344105    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.344170 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.344130    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.444899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.444873    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445040 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.444906    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445040 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.444943    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwfp5\" (UniqueName: \"kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445040 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.444982    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445040 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.445003    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445222 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.445072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.445421    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445562 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.445465    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.445562 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.445500    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.447410 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.447389    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.447616 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.447578    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.453375 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.453352    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwfp5\" (UniqueName: \"kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5\") pod \"test-model-4785d6-kserve-575d6cc4fc-76vzh\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") " pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.588035 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.587980    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:28.720344 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:28.720160    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:35:28.722707 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:35:28.722674    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e40f7f7_322d_4782_b5be_9b89739c51a3.slice/crio-917ab0e1d6e44d0579c0998de3b3140a54db4ed574449c5368c8f71e6895af29 WatchSource:0}: Error finding container 917ab0e1d6e44d0579c0998de3b3140a54db4ed574449c5368c8f71e6895af29: Status 404 returned error can't find the container with id 917ab0e1d6e44d0579c0998de3b3140a54db4ed574449c5368c8f71e6895af29
Jul 14 18:35:29.055499 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:29.055467    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" event={"ID":"4e40f7f7-322d-4782-b5be-9b89739c51a3","Type":"ContainerStarted","Data":"6906aee59b13aeb877e00f0fc857b8761217546c7e0d58c0e2c0a96085500337"}
Jul 14 18:35:29.055499 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:29.055504    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" event={"ID":"4e40f7f7-322d-4782-b5be-9b89739c51a3","Type":"ContainerStarted","Data":"917ab0e1d6e44d0579c0998de3b3140a54db4ed574449c5368c8f71e6895af29"}
Jul 14 18:35:29.077949 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:29.077904    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" podStartSLOduration=1.077889235 podStartE2EDuration="1.077889235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:35:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:35:29.075131857 +0000 UTC m=+2447.276687008" watchObservedRunningTime="2026-07-14 18:35:29.077889235 +0000 UTC m=+2447.279444374"
Jul 14 18:35:38.594450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:38.589196    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:38.594450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:38.589303    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:38.605409 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:38.605389    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:39.108891 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:39.108871    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:35:48.769765 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.769724    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:35:48.772996 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.772977    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.775637 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.775613    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-55e12d\"/\"default-dockercfg-hgwsh\""
Jul 14 18:35:48.775637 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.775622    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-55e12d\"/\"test-model-55e12d-kserve-self-signed-certs\""
Jul 14 18:35:48.775789 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.775656    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-55e12d\"/\"kube-root-ca.crt\""
Jul 14 18:35:48.775993 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.775971    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-55e12d\"/\"openshift-service-ca.crt\""
Jul 14 18:35:48.785049 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.785028    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:35:48.917786 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.917750    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.917943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.917803    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.917943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.917850    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.917943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.917873    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.918039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.917968    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:48.918039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:48.918000    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wh7n\" (UniqueName: \"kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018481 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018451    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018660 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018496    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018660 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018628    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018773 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018667    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018844 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018819    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018844 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018831    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.018949 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018881    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wh7n\" (UniqueName: \"kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.019010 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.018979    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.019052 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.019021    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.020983 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.020925    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.021173 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.021157    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.027964 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.027935    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wh7n\" (UniqueName: \"kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n\") pod \"test-model-55e12d-kserve-5577b9c8f-fkr64\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") " pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.082765 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.082739    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:49.215896 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:49.215860    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:35:49.218738 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:35:49.218706    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9cd707b5_ff3a_433b_b093_6d1e5eee3898.slice/crio-c75313b4f5afbe25edc92fa8de3059514507d9b4a8c58086ab7cefafc95e8eea WatchSource:0}: Error finding container c75313b4f5afbe25edc92fa8de3059514507d9b4a8c58086ab7cefafc95e8eea: Status 404 returned error can't find the container with id c75313b4f5afbe25edc92fa8de3059514507d9b4a8c58086ab7cefafc95e8eea
Jul 14 18:35:50.153700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:50.153659    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" event={"ID":"9cd707b5-ff3a-433b-b093-6d1e5eee3898","Type":"ContainerStarted","Data":"53e727d3b4a042d4091bbdc4fd82b483579728ecb85b0ce0e0e47dfc0ea8e47e"}
Jul 14 18:35:50.154063 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:50.153708    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" event={"ID":"9cd707b5-ff3a-433b-b093-6d1e5eee3898","Type":"ContainerStarted","Data":"c75313b4f5afbe25edc92fa8de3059514507d9b4a8c58086ab7cefafc95e8eea"}
Jul 14 18:35:50.172330 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:50.172286    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" podStartSLOduration=2.172271311 podStartE2EDuration="2.172271311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:35:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:35:50.171361099 +0000 UTC m=+2468.372916239" watchObservedRunningTime="2026-07-14 18:35:50.172271311 +0000 UTC m=+2468.373826464"
Jul 14 18:35:59.082915 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:59.082881    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:59.083284 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:59.082927    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:59.095019 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:59.094994    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:35:59.200926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:35:59.200903    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:36:10.153511 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:10.153481    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:36:11.100052 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:11.100022    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:36:21.743574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:21.743541    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:36:21.743940 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:21.743756    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" podUID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" containerName="maas-api" containerID="cri-o://5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b" gracePeriod=30
Jul 14 18:36:21.988068 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:21.988045    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:36:22.085874 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.085808    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca\") pod \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") "
Jul 14 18:36:22.085874 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.085851    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9lr5\" (UniqueName: \"kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5\") pod \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") "
Jul 14 18:36:22.086071 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.085877    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls\") pod \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\" (UID: \"fed9c65f-8f86-4df7-a560-60eec2e80ebf\") "
Jul 14 18:36:22.086130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.086102    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fed9c65f-8f86-4df7-a560-60eec2e80ebf" (UID: "fed9c65f-8f86-4df7-a560-60eec2e80ebf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:36:22.087814 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.087790    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5" (OuterVolumeSpecName: "kube-api-access-p9lr5") pod "fed9c65f-8f86-4df7-a560-60eec2e80ebf" (UID: "fed9c65f-8f86-4df7-a560-60eec2e80ebf"). InnerVolumeSpecName "kube-api-access-p9lr5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:36:22.087922 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.087873    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fed9c65f-8f86-4df7-a560-60eec2e80ebf" (UID: "fed9c65f-8f86-4df7-a560-60eec2e80ebf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:36:22.187128 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.187095    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fed9c65f-8f86-4df7-a560-60eec2e80ebf-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:22.187128 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.187121    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9lr5\" (UniqueName: \"kubernetes.io/projected/fed9c65f-8f86-4df7-a560-60eec2e80ebf-kube-api-access-p9lr5\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:22.187128 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.187131    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fed9c65f-8f86-4df7-a560-60eec2e80ebf-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:22.286503 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.286467    2556 generic.go:356] "Generic (PLEG): container finished" podID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" containerID="5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b" exitCode=0
Jul 14 18:36:22.286680 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.286509    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"
Jul 14 18:36:22.286680 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.286558    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" event={"ID":"fed9c65f-8f86-4df7-a560-60eec2e80ebf","Type":"ContainerDied","Data":"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"}
Jul 14 18:36:22.286680 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.286628    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp" event={"ID":"fed9c65f-8f86-4df7-a560-60eec2e80ebf","Type":"ContainerDied","Data":"9d5ddba111faf9dff143dae75dd4e580cb70e6a9c28566857a3fe1fc5f507aa4"}
Jul 14 18:36:22.286680 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.286650    2556 scope.go:117] "RemoveContainer" containerID="5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"
Jul 14 18:36:22.295546 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.295530    2556 scope.go:117] "RemoveContainer" containerID="5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"
Jul 14 18:36:22.295828 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:36:22.295809    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b\": container with ID starting with 5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b not found: ID does not exist" containerID="5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"
Jul 14 18:36:22.295908 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.295833    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b"} err="failed to get container status \"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b\": rpc error: code = NotFound desc = could not find container \"5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b\": container with ID starting with 5b6a180743ba6735dbd5e39c683c7f742ef4112a1f93fccc2c593f8303bf302b not found: ID does not exist"
Jul 14 18:36:22.309429 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.309405    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:36:22.312424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.312404    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-4785d6-c95876c98-754vp"]
Jul 14 18:36:22.324921 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:22.324895    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" path="/var/lib/kubelet/pods/fed9c65f-8f86-4df7-a560-60eec2e80ebf/volumes"
Jul 14 18:36:25.172924 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.172885    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" podUID="4e40f7f7-322d-4782-b5be-9b89739c51a3" containerName="main" containerID="cri-o://6906aee59b13aeb877e00f0fc857b8761217546c7e0d58c0e2c0a96085500337" gracePeriod=45
Jul 14 18:36:25.303628 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.303573    2556 generic.go:356] "Generic (PLEG): container finished" podID="4e40f7f7-322d-4782-b5be-9b89739c51a3" containerID="6906aee59b13aeb877e00f0fc857b8761217546c7e0d58c0e2c0a96085500337" exitCode=0
Jul 14 18:36:25.303628 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.303614    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" event={"ID":"4e40f7f7-322d-4782-b5be-9b89739c51a3","Type":"ContainerDied","Data":"6906aee59b13aeb877e00f0fc857b8761217546c7e0d58c0e2c0a96085500337"}
Jul 14 18:36:25.420889 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.420869    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:36:25.514958 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.514893    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.514958 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.514921    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.515161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.514965    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.515161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.514983    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.515161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515030    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.515161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515060    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwfp5\" (UniqueName: \"kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5\") pod \"4e40f7f7-322d-4782-b5be-9b89739c51a3\" (UID: \"4e40f7f7-322d-4782-b5be-9b89739c51a3\") "
Jul 14 18:36:25.515161 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515132    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:25.515424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515285    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home" (OuterVolumeSpecName: "home") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:25.515424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515303    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:25.515424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515416    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:25.515770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515430    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:25.515770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.515438    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:25.517133 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.517103    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm" (OuterVolumeSpecName: "dshm") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:25.517225 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.517160    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:36:25.517266 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.517248    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5" (OuterVolumeSpecName: "kube-api-access-pwfp5") pod "4e40f7f7-322d-4782-b5be-9b89739c51a3" (UID: "4e40f7f7-322d-4782-b5be-9b89739c51a3"). InnerVolumeSpecName "kube-api-access-pwfp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:36:25.616624 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.616599    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwfp5\" (UniqueName: \"kubernetes.io/projected/4e40f7f7-322d-4782-b5be-9b89739c51a3-kube-api-access-pwfp5\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:25.616624 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.616621    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e40f7f7-322d-4782-b5be-9b89739c51a3-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:25.616745 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:25.616636    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e40f7f7-322d-4782-b5be-9b89739c51a3-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.119346 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.119310    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" podUID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" containerName="main" containerID="cri-o://53e727d3b4a042d4091bbdc4fd82b483579728ecb85b0ce0e0e47dfc0ea8e47e" gracePeriod=45
Jul 14 18:36:26.311499 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.311471    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"
Jul 14 18:36:26.311499 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.311483    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh" event={"ID":"4e40f7f7-322d-4782-b5be-9b89739c51a3","Type":"ContainerDied","Data":"917ab0e1d6e44d0579c0998de3b3140a54db4ed574449c5368c8f71e6895af29"}
Jul 14 18:36:26.311979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.311536    2556 scope.go:117] "RemoveContainer" containerID="6906aee59b13aeb877e00f0fc857b8761217546c7e0d58c0e2c0a96085500337"
Jul 14 18:36:26.313399 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.313366    2556 generic.go:356] "Generic (PLEG): container finished" podID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" containerID="53e727d3b4a042d4091bbdc4fd82b483579728ecb85b0ce0e0e47dfc0ea8e47e" exitCode=0
Jul 14 18:36:26.320413 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.320384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" event={"ID":"9cd707b5-ff3a-433b-b093-6d1e5eee3898","Type":"ContainerDied","Data":"53e727d3b4a042d4091bbdc4fd82b483579728ecb85b0ce0e0e47dfc0ea8e47e"}
Jul 14 18:36:26.337768 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.337742    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:36:26.341054 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.341033    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4785d6/test-model-4785d6-kserve-575d6cc4fc-76vzh"]
Jul 14 18:36:26.369919 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.369874    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:36:26.422872 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.422848    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wh7n\" (UniqueName: \"kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.422973 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.422884    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.422973 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.422905    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.423039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.422977    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.423039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423008    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.423098 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423086    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs\") pod \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\" (UID: \"9cd707b5-ff3a-433b-b093-6d1e5eee3898\") "
Jul 14 18:36:26.423164 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423141    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home" (OuterVolumeSpecName: "home") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:26.423201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423155    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:26.423296 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423274    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache" (OuterVolumeSpecName: "model-cache") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:26.423396 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423380    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.423450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423403    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.423450 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.423413    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.425202 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.425180    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:36:26.425286 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.425233    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm" (OuterVolumeSpecName: "dshm") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:36:26.425330 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.425283    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n" (OuterVolumeSpecName: "kube-api-access-7wh7n") pod "9cd707b5-ff3a-433b-b093-6d1e5eee3898" (UID: "9cd707b5-ff3a-433b-b093-6d1e5eee3898"). InnerVolumeSpecName "kube-api-access-7wh7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:36:26.523959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.523919    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cd707b5-ff3a-433b-b093-6d1e5eee3898-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.523959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.523955    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wh7n\" (UniqueName: \"kubernetes.io/projected/9cd707b5-ff3a-433b-b093-6d1e5eee3898-kube-api-access-7wh7n\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:26.523959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:26.523966    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cd707b5-ff3a-433b-b093-6d1e5eee3898-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:27.319809 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:27.319775    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"
Jul 14 18:36:27.320267 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:27.319827    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64" event={"ID":"9cd707b5-ff3a-433b-b093-6d1e5eee3898","Type":"ContainerDied","Data":"c75313b4f5afbe25edc92fa8de3059514507d9b4a8c58086ab7cefafc95e8eea"}
Jul 14 18:36:27.320267 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:27.319876    2556 scope.go:117] "RemoveContainer" containerID="53e727d3b4a042d4091bbdc4fd82b483579728ecb85b0ce0e0e47dfc0ea8e47e"
Jul 14 18:36:27.345201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:27.345168    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:36:27.348300 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:27.348276    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-55e12d/test-model-55e12d-kserve-5577b9c8f-fkr64"]
Jul 14 18:36:28.319771 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:28.319739    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e40f7f7-322d-4782-b5be-9b89739c51a3" path="/var/lib/kubelet/pods/4e40f7f7-322d-4782-b5be-9b89739c51a3/volumes"
Jul 14 18:36:28.320197 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:28.320178    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" path="/var/lib/kubelet/pods/9cd707b5-ff3a-433b-b093-6d1e5eee3898/volumes"
Jul 14 18:36:48.078793 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.078754    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:36:48.079286 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.079021    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" podUID="176262ec-a4f8-4b1b-8578-16079a61c4c0" containerName="maas-api" containerID="cri-o://c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442" gracePeriod=30
Jul 14 18:36:48.318419 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.318395    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:36:48.389049 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.389023    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca\") pod \"176262ec-a4f8-4b1b-8578-16079a61c4c0\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") "
Jul 14 18:36:48.389231 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.389083    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls\") pod \"176262ec-a4f8-4b1b-8578-16079a61c4c0\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") "
Jul 14 18:36:48.389231 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.389134    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fllvx\" (UniqueName: \"kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx\") pod \"176262ec-a4f8-4b1b-8578-16079a61c4c0\" (UID: \"176262ec-a4f8-4b1b-8578-16079a61c4c0\") "
Jul 14 18:36:48.389454 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.389426    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "176262ec-a4f8-4b1b-8578-16079a61c4c0" (UID: "176262ec-a4f8-4b1b-8578-16079a61c4c0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:36:48.391392 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.391358    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx" (OuterVolumeSpecName: "kube-api-access-fllvx") pod "176262ec-a4f8-4b1b-8578-16079a61c4c0" (UID: "176262ec-a4f8-4b1b-8578-16079a61c4c0"). InnerVolumeSpecName "kube-api-access-fllvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:36:48.391392 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.391367    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "176262ec-a4f8-4b1b-8578-16079a61c4c0" (UID: "176262ec-a4f8-4b1b-8578-16079a61c4c0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:36:48.407014 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.406987    2556 generic.go:356] "Generic (PLEG): container finished" podID="176262ec-a4f8-4b1b-8578-16079a61c4c0" containerID="c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442" exitCode=0
Jul 14 18:36:48.407103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.407016    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" event={"ID":"176262ec-a4f8-4b1b-8578-16079a61c4c0","Type":"ContainerDied","Data":"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"}
Jul 14 18:36:48.407103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.407035    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"
Jul 14 18:36:48.407103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.407069    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j" event={"ID":"176262ec-a4f8-4b1b-8578-16079a61c4c0","Type":"ContainerDied","Data":"ed0cb1d3d5c1cec62d150e383846dad749030dfac26d845a54ebf7355d4b75c9"}
Jul 14 18:36:48.407103 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.407090    2556 scope.go:117] "RemoveContainer" containerID="c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"
Jul 14 18:36:48.416080 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.416063    2556 scope.go:117] "RemoveContainer" containerID="c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"
Jul 14 18:36:48.416330 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:36:48.416309    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442\": container with ID starting with c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442 not found: ID does not exist" containerID="c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"
Jul 14 18:36:48.416397 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.416337    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442"} err="failed to get container status \"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442\": rpc error: code = NotFound desc = could not find container \"c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442\": container with ID starting with c1d8f823953a69fab43fc79689e09b8a17bc908c50812c12d4dc343554dcd442 not found: ID does not exist"
Jul 14 18:36:48.429567 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.429542    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:36:48.436430 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.436410    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-55e12d-7cfdf5d96c-5dw5j"]
Jul 14 18:36:48.490515 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.490496    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fllvx\" (UniqueName: \"kubernetes.io/projected/176262ec-a4f8-4b1b-8578-16079a61c4c0-kube-api-access-fllvx\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:48.490515 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.490515    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/176262ec-a4f8-4b1b-8578-16079a61c4c0-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:48.490650 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:48.490526    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/176262ec-a4f8-4b1b-8578-16079a61c4c0-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:36:50.318734 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:36:50.318701    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="176262ec-a4f8-4b1b-8578-16079a61c4c0" path="/var/lib/kubelet/pods/176262ec-a4f8-4b1b-8578-16079a61c4c0/volumes"
Jul 14 18:37:08.939313 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939275    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:08.939734 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939717    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="176262ec-a4f8-4b1b-8578-16079a61c4c0" containerName="maas-api"
Jul 14 18:37:08.939778 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939738    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="176262ec-a4f8-4b1b-8578-16079a61c4c0" containerName="maas-api"
Jul 14 18:37:08.939778 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939754    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" containerName="main"
Jul 14 18:37:08.939778 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939763    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" containerName="main"
Jul 14 18:37:08.939861 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939797    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" containerName="maas-api"
Jul 14 18:37:08.939861 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939806    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" containerName="maas-api"
Jul 14 18:37:08.939861 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939817    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e40f7f7-322d-4782-b5be-9b89739c51a3" containerName="main"
Jul 14 18:37:08.939861 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939825    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e40f7f7-322d-4782-b5be-9b89739c51a3" containerName="main"
Jul 14 18:37:08.939972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939906    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e40f7f7-322d-4782-b5be-9b89739c51a3" containerName="main"
Jul 14 18:37:08.939972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939920    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cd707b5-ff3a-433b-b093-6d1e5eee3898" containerName="main"
Jul 14 18:37:08.939972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939931    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="fed9c65f-8f86-4df7-a560-60eec2e80ebf" containerName="maas-api"
Jul 14 18:37:08.939972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.939940    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="176262ec-a4f8-4b1b-8578-16079a61c4c0" containerName="maas-api"
Jul 14 18:37:08.944322 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.944301    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:08.946994 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.946973    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-e0452a-serving-cert\""
Jul 14 18:37:08.947219 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.947031    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:37:08.947376 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.947031    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:37:08.947532 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.947507    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:37:08.954922 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:08.954901    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:09.042973 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.042951    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.043101 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.042998    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.043101 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.043091    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k224\" (UniqueName: \"kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.143520 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.143491    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k224\" (UniqueName: \"kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.143668 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.143545    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.143668 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.143611    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.144208 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.144186    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.146020 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.145999    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.152238 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.152216    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k224\" (UniqueName: \"kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224\") pod \"maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.256407 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.256361    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:09.386684 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.386652    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:09.389493 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:37:09.389455    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b58d9e_ba80_45b9_bbbc_2a89b8ae4736.slice/crio-f5ef1bfdf19ff7ecb665c2106c2c0544961d2cf10dd9505e12a996c7028b461d WatchSource:0}: Error finding container f5ef1bfdf19ff7ecb665c2106c2c0544961d2cf10dd9505e12a996c7028b461d: Status 404 returned error can't find the container with id f5ef1bfdf19ff7ecb665c2106c2c0544961d2cf10dd9505e12a996c7028b461d
Jul 14 18:37:09.390904 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.390877    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 18:37:09.491957 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:09.491933    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" event={"ID":"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736","Type":"ContainerStarted","Data":"f5ef1bfdf19ff7ecb665c2106c2c0544961d2cf10dd9505e12a996c7028b461d"}
Jul 14 18:37:10.498398 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:10.498366    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" event={"ID":"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736","Type":"ContainerStarted","Data":"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"}
Jul 14 18:37:10.498758 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:10.498442    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:10.521972 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:10.521928    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" podStartSLOduration=1.6383188 podStartE2EDuration="2.521912513s" totalImagesPullingTime="883.593713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:37:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:37:10.520869725 +0000 UTC m=+2548.722424865" watchObservedRunningTime="2026-07-14 18:37:10.521912513 +0000 UTC m=+2548.723467706"
Jul 14 18:37:16.508353 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:16.508324    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:16.869359 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:16.869326    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:37:16.872844 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:16.872820    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:16.875476 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:16.875451    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-b418a4-serving-cert\""
Jul 14 18:37:16.887765 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:16.887740    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:37:17.005490 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.005461    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.005669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.005570    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44thz\" (UniqueName: \"kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.005669 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.005639    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.106281 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.106246    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.106442 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.106293    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.106442 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.106363    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44thz\" (UniqueName: \"kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.106442 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:37:17.106418    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-serving-cert: secret "maas-api-e2e-api-b-b418a4-serving-cert" not found
Jul 14 18:37:17.106624 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:37:17.106495    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls podName:dabc972f-9a99-4928-87ad-b9d8e9090f79 nodeName:}" failed. No retries permitted until 2026-07-14 18:37:17.60647334 +0000 UTC m=+2555.808028461 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls") pod "maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" (UID: "dabc972f-9a99-4928-87ad-b9d8e9090f79") : secret "maas-api-e2e-api-b-b418a4-serving-cert" not found
Jul 14 18:37:17.107003 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.106985    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.119154 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.119134    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44thz\" (UniqueName: \"kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.611514 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.611475    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.613880 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.613859    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") pod \"maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.783442 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.783415    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:17.915875 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:17.915851    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:37:17.917864 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:37:17.917834    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddabc972f_9a99_4928_87ad_b9d8e9090f79.slice/crio-25313bb884963b0aa890cac1ee1ea8df51c697b74cfaef21a4e18b685ab4078c WatchSource:0}: Error finding container 25313bb884963b0aa890cac1ee1ea8df51c697b74cfaef21a4e18b685ab4078c: Status 404 returned error can't find the container with id 25313bb884963b0aa890cac1ee1ea8df51c697b74cfaef21a4e18b685ab4078c
Jul 14 18:37:18.539545 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:18.539457    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" event={"ID":"dabc972f-9a99-4928-87ad-b9d8e9090f79","Type":"ContainerStarted","Data":"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"}
Jul 14 18:37:18.539545 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:18.539510    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" event={"ID":"dabc972f-9a99-4928-87ad-b9d8e9090f79","Type":"ContainerStarted","Data":"25313bb884963b0aa890cac1ee1ea8df51c697b74cfaef21a4e18b685ab4078c"}
Jul 14 18:37:18.539796 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:18.539551    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:18.560329 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:18.560282    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" podStartSLOduration=2.257646562 podStartE2EDuration="2.560266985s" totalImagesPullingTime="302.620423ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:37:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:37:18.558329831 +0000 UTC m=+2556.759884973" watchObservedRunningTime="2026-07-14 18:37:18.560266985 +0000 UTC m=+2556.761822123"
Jul 14 18:37:24.549203 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:24.549169    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:37:40.325515 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.325484    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:40.325990 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.325717    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" podUID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" containerName="maas-api" containerID="cri-o://39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60" gracePeriod=30
Jul 14 18:37:40.560940 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.560919    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:40.591552 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.591499    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca\") pod \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") "
Jul 14 18:37:40.591552 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.591543    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls\") pod \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") "
Jul 14 18:37:40.591711 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.591606    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k224\" (UniqueName: \"kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224\") pod \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\" (UID: \"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736\") "
Jul 14 18:37:40.591940 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.591909    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" (UID: "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:37:40.593705 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.593670    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224" (OuterVolumeSpecName: "kube-api-access-6k224") pod "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" (UID: "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736"). InnerVolumeSpecName "kube-api-access-6k224". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:37:40.593799 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.593735    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" (UID: "04b58d9e-ba80-45b9-bbbc-2a89b8ae4736"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:37:40.640706 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.640677    2556 generic.go:356] "Generic (PLEG): container finished" podID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" containerID="39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60" exitCode=0
Jul 14 18:37:40.640793 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.640720    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"
Jul 14 18:37:40.640793 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.640766    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" event={"ID":"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736","Type":"ContainerDied","Data":"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"}
Jul 14 18:37:40.640864 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.640808    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms" event={"ID":"04b58d9e-ba80-45b9-bbbc-2a89b8ae4736","Type":"ContainerDied","Data":"f5ef1bfdf19ff7ecb665c2106c2c0544961d2cf10dd9505e12a996c7028b461d"}
Jul 14 18:37:40.640864 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.640831    2556 scope.go:117] "RemoveContainer" containerID="39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"
Jul 14 18:37:40.649918 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.649900    2556 scope.go:117] "RemoveContainer" containerID="39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"
Jul 14 18:37:40.650157 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:37:40.650132    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60\": container with ID starting with 39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60 not found: ID does not exist" containerID="39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"
Jul 14 18:37:40.650215 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.650166    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60"} err="failed to get container status \"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60\": rpc error: code = NotFound desc = could not find container \"39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60\": container with ID starting with 39cdc87b2bdaf514c3d75f407679af6139bce1a8a4dae581c7d0a43d3b0a0e60 not found: ID does not exist"
Jul 14 18:37:40.663383 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.663359    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:40.667847 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.667826    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-e0452a-598dcc8ccb-phmms"]
Jul 14 18:37:40.692838 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.692815    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:37:40.692838 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.692835    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:37:40.692953 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:40.692844    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k224\" (UniqueName: \"kubernetes.io/projected/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736-kube-api-access-6k224\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:37:42.319860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:37:42.319829    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" path="/var/lib/kubelet/pods/04b58d9e-ba80-45b9-bbbc-2a89b8ae4736/volumes"
Jul 14 18:38:06.071729 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.071627    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:38:06.072300 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.071846    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" podUID="dabc972f-9a99-4928-87ad-b9d8e9090f79" containerName="maas-api" containerID="cri-o://5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d" gracePeriod=30
Jul 14 18:38:06.317933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.317912    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:38:06.395715 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.395685    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") pod \"dabc972f-9a99-4928-87ad-b9d8e9090f79\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") "
Jul 14 18:38:06.395851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.395736    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca\") pod \"dabc972f-9a99-4928-87ad-b9d8e9090f79\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") "
Jul 14 18:38:06.395851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.395819    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44thz\" (UniqueName: \"kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz\") pod \"dabc972f-9a99-4928-87ad-b9d8e9090f79\" (UID: \"dabc972f-9a99-4928-87ad-b9d8e9090f79\") "
Jul 14 18:38:06.396106 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.396082    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dabc972f-9a99-4928-87ad-b9d8e9090f79" (UID: "dabc972f-9a99-4928-87ad-b9d8e9090f79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:38:06.397735 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.397711    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dabc972f-9a99-4928-87ad-b9d8e9090f79" (UID: "dabc972f-9a99-4928-87ad-b9d8e9090f79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:38:06.397850 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.397833    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz" (OuterVolumeSpecName: "kube-api-access-44thz") pod "dabc972f-9a99-4928-87ad-b9d8e9090f79" (UID: "dabc972f-9a99-4928-87ad-b9d8e9090f79"). InnerVolumeSpecName "kube-api-access-44thz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:38:06.496783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.496762    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44thz\" (UniqueName: \"kubernetes.io/projected/dabc972f-9a99-4928-87ad-b9d8e9090f79-kube-api-access-44thz\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:38:06.496783 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.496782    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dabc972f-9a99-4928-87ad-b9d8e9090f79-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:38:06.496899 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.496790    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dabc972f-9a99-4928-87ad-b9d8e9090f79-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:38:06.751323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.751268    2556 generic.go:356] "Generic (PLEG): container finished" podID="dabc972f-9a99-4928-87ad-b9d8e9090f79" containerID="5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d" exitCode=0
Jul 14 18:38:06.751323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.751318    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"
Jul 14 18:38:06.751451 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.751345    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" event={"ID":"dabc972f-9a99-4928-87ad-b9d8e9090f79","Type":"ContainerDied","Data":"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"}
Jul 14 18:38:06.751451 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.751384    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf" event={"ID":"dabc972f-9a99-4928-87ad-b9d8e9090f79","Type":"ContainerDied","Data":"25313bb884963b0aa890cac1ee1ea8df51c697b74cfaef21a4e18b685ab4078c"}
Jul 14 18:38:06.751451 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.751404    2556 scope.go:117] "RemoveContainer" containerID="5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"
Jul 14 18:38:06.760770 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.760755    2556 scope.go:117] "RemoveContainer" containerID="5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"
Jul 14 18:38:06.761022 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:06.761005    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d\": container with ID starting with 5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d not found: ID does not exist" containerID="5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"
Jul 14 18:38:06.761068 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.761030    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d"} err="failed to get container status \"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d\": rpc error: code = NotFound desc = could not find container \"5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d\": container with ID starting with 5c6623d553c430812d9d198032681db7d03beb3008de9fbce9e56789a219390d not found: ID does not exist"
Jul 14 18:38:06.784605 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.784570    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:38:06.789698 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:06.789679    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-b418a4-8c876bf78-8zsrf"]
Jul 14 18:38:08.318888 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:08.318846    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dabc972f-9a99-4928-87ad-b9d8e9090f79" path="/var/lib/kubelet/pods/dabc972f-9a99-4928-87ad-b9d8e9090f79/volumes"
Jul 14 18:38:25.301566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301535    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301842    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dabc972f-9a99-4928-87ad-b9d8e9090f79" containerName="maas-api"
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301855    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="dabc972f-9a99-4928-87ad-b9d8e9090f79" containerName="maas-api"
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301875    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" containerName="maas-api"
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301880    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" containerName="maas-api"
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301935    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="dabc972f-9a99-4928-87ad-b9d8e9090f79" containerName="maas-api"
Jul 14 18:38:25.302130 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.301943    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b58d9e-ba80-45b9-bbbc-2a89b8ae4736" containerName="maas-api"
Jul 14 18:38:25.304809 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.304788    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.308441 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.308409    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 14 18:38:25.308561 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.308542    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dzj5k\""
Jul 14 18:38:25.309139 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.309119    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 14 18:38:25.309615 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.309599    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-7e7ccb-serving-cert\""
Jul 14 18:38:25.322194 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.322169    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:38:25.446752 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.446721    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.446923 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.446775    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.446923 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.446819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2qc8\" (UniqueName: \"kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.547304 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.547271    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.547485 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.547316    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.547485 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.547345    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2qc8\" (UniqueName: \"kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.547485 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:25.547470    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-serving-cert: secret "maas-api-e2e-shared-a-7e7ccb-serving-cert" not found
Jul 14 18:38:25.547697 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:25.547543    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls podName:5b307d03-8f40-4d92-80dc-247326fc0c1e nodeName:}" failed. No retries permitted until 2026-07-14 18:38:26.047522963 +0000 UTC m=+2624.249078094 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls") pod "maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" (UID: "5b307d03-8f40-4d92-80dc-247326fc0c1e") : secret "maas-api-e2e-shared-a-7e7ccb-serving-cert" not found
Jul 14 18:38:25.548114 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.548089    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:25.561913 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:25.561860    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2qc8\" (UniqueName: \"kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:26.051767 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:26.051732    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:26.051958 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:26.051904    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-serving-cert: secret "maas-api-e2e-shared-a-7e7ccb-serving-cert" not found
Jul 14 18:38:26.052031 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:26.051976    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls podName:5b307d03-8f40-4d92-80dc-247326fc0c1e nodeName:}" failed. No retries permitted until 2026-07-14 18:38:27.05195691 +0000 UTC m=+2625.253512040 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls") pod "maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" (UID: "5b307d03-8f40-4d92-80dc-247326fc0c1e") : secret "maas-api-e2e-shared-a-7e7ccb-serving-cert" not found
Jul 14 18:38:27.060507 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.060467    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:27.062799 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.062780    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:27.117637 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.117611    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:27.259203 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.259175    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:38:27.260901 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:38:27.260874    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b307d03_8f40_4d92_80dc_247326fc0c1e.slice/crio-71b13b02e87463b18e0de18fc11480c73f2b1a580c1b45e1e1ba013ea9be8141 WatchSource:0}: Error finding container 71b13b02e87463b18e0de18fc11480c73f2b1a580c1b45e1e1ba013ea9be8141: Status 404 returned error can't find the container with id 71b13b02e87463b18e0de18fc11480c73f2b1a580c1b45e1e1ba013ea9be8141
Jul 14 18:38:27.841417 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.841392    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" event={"ID":"5b307d03-8f40-4d92-80dc-247326fc0c1e","Type":"ContainerStarted","Data":"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"}
Jul 14 18:38:27.841545 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.841437    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" event={"ID":"5b307d03-8f40-4d92-80dc-247326fc0c1e","Type":"ContainerStarted","Data":"71b13b02e87463b18e0de18fc11480c73f2b1a580c1b45e1e1ba013ea9be8141"}
Jul 14 18:38:27.841545 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.841504    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:27.864722 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:27.864678    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" podStartSLOduration=2.352946457 podStartE2EDuration="2.8646609s" totalImagesPullingTime="511.714443ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:38:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:38:27.863176592 +0000 UTC m=+2626.064731731" watchObservedRunningTime="2026-07-14 18:38:27.8646609 +0000 UTC m=+2626.066216043"
Jul 14 18:38:33.371943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.371899    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:38:33.375285 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.375263    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.383248 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:33.383222    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.383365 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:33.383341    2556 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-shared-b-b860a5-serving-cert\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b860a5-serving-cert\"" type="*v1.Secret"
Jul 14 18:38:33.401958 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.401929    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:38:33.511127 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.511104    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sc89\" (UniqueName: \"kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.511226 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.511177    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.511226 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.511209    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.611525 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.611485    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.611673 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.611544    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.611673 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.611610    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sc89\" (UniqueName: \"kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.612074 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.612055    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.621098 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.621076    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sc89\" (UniqueName: \"kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:33.851299 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:33.851275    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:38:34.612234 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:34.612202    2556 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-serving-cert: failed to sync secret cache: timed out waiting for the condition
Jul 14 18:38:34.612621 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:38:34.612272    2556 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls podName:0acddc6d-12ec-4b3a-bcca-08dba795a99b nodeName:}" failed. No retries permitted until 2026-07-14 18:38:35.112257716 +0000 UTC m=+2633.313812834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls") pod "maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" (UID: "0acddc6d-12ec-4b3a-bcca-08dba795a99b") : failed to sync secret cache: timed out waiting for the condition
Jul 14 18:38:34.786385 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:34.786360    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-b860a5-serving-cert\""
Jul 14 18:38:35.120697 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.120670    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:35.123032 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.123005    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") pod \"maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:35.184870 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.184842    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:35.316766 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.316731    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:38:35.320530 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:38:35.320499    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0acddc6d_12ec_4b3a_bcca_08dba795a99b.slice/crio-77e36bb9213a87519ce1c2d8425c9a8ae9d27bc1cc5438690c9f07bb6b84a02e WatchSource:0}: Error finding container 77e36bb9213a87519ce1c2d8425c9a8ae9d27bc1cc5438690c9f07bb6b84a02e: Status 404 returned error can't find the container with id 77e36bb9213a87519ce1c2d8425c9a8ae9d27bc1cc5438690c9f07bb6b84a02e
Jul 14 18:38:35.878944 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.878908    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" event={"ID":"0acddc6d-12ec-4b3a-bcca-08dba795a99b","Type":"ContainerStarted","Data":"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"}
Jul 14 18:38:35.879390 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.878975    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" event={"ID":"0acddc6d-12ec-4b3a-bcca-08dba795a99b","Type":"ContainerStarted","Data":"77e36bb9213a87519ce1c2d8425c9a8ae9d27bc1cc5438690c9f07bb6b84a02e"}
Jul 14 18:38:35.879390 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.879005    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:35.901825 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:35.901779    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" podStartSLOduration=2.558478688 podStartE2EDuration="2.901763893s" totalImagesPullingTime="343.285205ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:38:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:38:35.900390047 +0000 UTC m=+2634.101945191" watchObservedRunningTime="2026-07-14 18:38:35.901763893 +0000 UTC m=+2634.103319032"
Jul 14 18:38:41.888609 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:41.888553    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:38:49.071529 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.071495    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:38:49.074560 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.074540    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.077571 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.077544    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e7ccb\"/\"auth-test-model-7e7ccb-kserve-self-signed-certs\""
Jul 14 18:38:49.077688 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.077576    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e7ccb\"/\"kube-root-ca.crt\""
Jul 14 18:38:49.077688 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.077605    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-7e7ccb\"/\"openshift-service-ca.crt\""
Jul 14 18:38:49.077833 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.077815    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-7e7ccb\"/\"default-dockercfg-2kjkj\""
Jul 14 18:38:49.087022 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.087002    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:38:49.125844 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.125819    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.125938 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.125849    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.125938 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.125881    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc5bn\" (UniqueName: \"kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.126019 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.125954    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.126019 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.125987    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.126084 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.126033    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227001 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.226983    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227008    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227031    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc5bn\" (UniqueName: \"kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227109 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227053    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227268 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227165    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227268 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227239    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227478 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227457    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227616 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227530    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.227616 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.227565    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.229389 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.229371    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.229549 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.229534    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.241889 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.241862    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc5bn\" (UniqueName: \"kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn\") pod \"auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") " pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.383718 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.383689    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:49.515871 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.515846    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:38:49.518239 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:38:49.518212    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod985b6b3a_bd50_4990_bb9c_0340d3cb0040.slice/crio-dddc4aa36d4c4ae9f9b3f8f0735e09ae21df7df74768aaa88be8b24d4277450d WatchSource:0}: Error finding container dddc4aa36d4c4ae9f9b3f8f0735e09ae21df7df74768aaa88be8b24d4277450d: Status 404 returned error can't find the container with id dddc4aa36d4c4ae9f9b3f8f0735e09ae21df7df74768aaa88be8b24d4277450d
Jul 14 18:38:49.938794 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.938763    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" event={"ID":"985b6b3a-bd50-4990-bb9c-0340d3cb0040","Type":"ContainerStarted","Data":"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"}
Jul 14 18:38:49.938960 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.938808    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" event={"ID":"985b6b3a-bd50-4990-bb9c-0340d3cb0040","Type":"ContainerStarted","Data":"dddc4aa36d4c4ae9f9b3f8f0735e09ae21df7df74768aaa88be8b24d4277450d"}
Jul 14 18:38:49.957076 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:49.957030    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" podStartSLOduration=0.957015237 podStartE2EDuration="957.015237ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:38:49.955302384 +0000 UTC m=+2648.156857524" watchObservedRunningTime="2026-07-14 18:38:49.957015237 +0000 UTC m=+2648.158570399"
Jul 14 18:38:59.384917 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:59.384883    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:59.384917 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:59.384923    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:59.398879 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:59.398851    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:38:59.991813 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:38:59.991786    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:39:08.199307 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.199270    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:39:08.203927 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.203911    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.208028 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.208000    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"default-dockercfg-46zf8\""
Jul 14 18:39:08.208153 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.208014    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"auth-test-model-b860a5-kserve-self-signed-certs\""
Jul 14 18:39:08.208153 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.208089    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"openshift-service-ca.crt\""
Jul 14 18:39:08.208273 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.208258    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"kube-root-ca.crt\""
Jul 14 18:39:08.216014 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.215989    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:39:08.274653 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274628    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.274799 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274666    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.274799 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274688    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.274799 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274784    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.274946 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274809    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.274946 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.274828    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88qmw\" (UniqueName: \"kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376030 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376004    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376174 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376038    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376174 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376058    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376174 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376174 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376174 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376118    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88qmw\" (UniqueName: \"kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376517 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376490    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376645 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376555    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.376645 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.376564    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.378519 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.378497    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.378641 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.378625    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.403693 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.403667    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88qmw\" (UniqueName: \"kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw\") pod \"auth-test-model-b860a5-kserve-75d749575f-gtt9r\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") " pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.513471 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.513404    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:08.647323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:08.647298    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:39:08.649386 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:39:08.649351    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfad2db2_f354_40a0_b2ef_5dc9d6426b4b.slice/crio-58e59414f15fc82992c60f3ed230fe748793fb1dc1d7eab0de2cde8f9b85afe4 WatchSource:0}: Error finding container 58e59414f15fc82992c60f3ed230fe748793fb1dc1d7eab0de2cde8f9b85afe4: Status 404 returned error can't find the container with id 58e59414f15fc82992c60f3ed230fe748793fb1dc1d7eab0de2cde8f9b85afe4
Jul 14 18:39:09.022223 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:09.022186    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" event={"ID":"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b","Type":"ContainerStarted","Data":"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"}
Jul 14 18:39:09.022387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:09.022233    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" event={"ID":"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b","Type":"ContainerStarted","Data":"58e59414f15fc82992c60f3ed230fe748793fb1dc1d7eab0de2cde8f9b85afe4"}
Jul 14 18:39:09.044301 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:09.044258    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" podStartSLOduration=1.04424532 podStartE2EDuration="1.04424532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:39:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:39:09.042891175 +0000 UTC m=+2667.244446315" watchObservedRunningTime="2026-07-14 18:39:09.04424532 +0000 UTC m=+2667.245800460"
Jul 14 18:39:18.513838 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:18.513796    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:18.514242 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:18.513863    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:18.526492 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:18.526461    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:19.076035 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:19.076010    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:39:42.335511 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:42.335397    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:39:42.348433 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:42.348407    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:39:42.348858 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:42.348839    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:39:42.355981 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:39:42.355959    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:41:03.147264 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:03.147182    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:41:04.085926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:04.085898    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:41:18.167085 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.167033    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" podUID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" containerName="main" containerID="cri-o://660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba" gracePeriod=45
Jul 14 18:41:18.405514 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.405492    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:41:18.516993 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.516918    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517120 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517002    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517120 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517040    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517120 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517065    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc5bn\" (UniqueName: \"kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517120 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517087    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517166    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache\") pod \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\" (UID: \"985b6b3a-bd50-4990-bb9c-0340d3cb0040\") "
Jul 14 18:41:18.517323 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517279    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:18.517415 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517373    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home" (OuterVolumeSpecName: "home") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:18.517471 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517430    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:18.517697 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.517625    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache" (OuterVolumeSpecName: "model-cache") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:18.519122 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.519102    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:41:18.519186 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.519150    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn" (OuterVolumeSpecName: "kube-api-access-vc5bn") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "kube-api-access-vc5bn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:41:18.519246 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.519229    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm" (OuterVolumeSpecName: "dshm") pod "985b6b3a-bd50-4990-bb9c-0340d3cb0040" (UID: "985b6b3a-bd50-4990-bb9c-0340d3cb0040"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:18.584480 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.584453    2556 generic.go:356] "Generic (PLEG): container finished" podID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" containerID="660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba" exitCode=0
Jul 14 18:41:18.584566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.584487    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" event={"ID":"985b6b3a-bd50-4990-bb9c-0340d3cb0040","Type":"ContainerDied","Data":"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"}
Jul 14 18:41:18.584566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.584493    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"
Jul 14 18:41:18.584566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.584511    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b" event={"ID":"985b6b3a-bd50-4990-bb9c-0340d3cb0040","Type":"ContainerDied","Data":"dddc4aa36d4c4ae9f9b3f8f0735e09ae21df7df74768aaa88be8b24d4277450d"}
Jul 14 18:41:18.584566 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.584528    2556 scope.go:117] "RemoveContainer" containerID="660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"
Jul 14 18:41:18.593211 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.593191    2556 scope.go:117] "RemoveContainer" containerID="660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"
Jul 14 18:41:18.593461 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:41:18.593444    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba\": container with ID starting with 660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba not found: ID does not exist" containerID="660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"
Jul 14 18:41:18.593507 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.593469    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba"} err="failed to get container status \"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba\": rpc error: code = NotFound desc = could not find container \"660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba\": container with ID starting with 660488d7e70956af55c666eb2a03403c8135a0cdd1af894f7bea971ee6ace6ba not found: ID does not exist"
Jul 14 18:41:18.605100 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.605079    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:41:18.609993 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.609973    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-7e7ccb/auth-test-model-7e7ccb-kserve-db47b9fc5-9rx5b"]
Jul 14 18:41:18.618823 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.618806    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:18.618883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.618839    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/985b6b3a-bd50-4990-bb9c-0340d3cb0040-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:18.618883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.618848    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:18.618883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.618857    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc5bn\" (UniqueName: \"kubernetes.io/projected/985b6b3a-bd50-4990-bb9c-0340d3cb0040-kube-api-access-vc5bn\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:18.618883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:18.618867    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/985b6b3a-bd50-4990-bb9c-0340d3cb0040-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.105394 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.105362    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" podUID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" containerName="main" containerID="cri-o://ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c" gracePeriod=45
Jul 14 18:41:19.345957 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.345929    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:41:19.425635 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425560    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88qmw\" (UniqueName: \"kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.425635 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425618    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.425860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425639    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.425860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425679    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.425860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425723    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.425860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425738    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir\") pod \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\" (UID: \"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b\") "
Jul 14 18:41:19.426061 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425909    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home" (OuterVolumeSpecName: "home") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:19.426061 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.425960    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache" (OuterVolumeSpecName: "model-cache") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:19.426148 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.426056    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:19.427867 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.427844    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm" (OuterVolumeSpecName: "dshm") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:41:19.427943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.427923    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw" (OuterVolumeSpecName: "kube-api-access-88qmw") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "kube-api-access-88qmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:41:19.427989 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.427939    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" (UID: "bfad2db2-f354-40a0-b2ef-5dc9d6426b4b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:41:19.526659 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526630    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88qmw\" (UniqueName: \"kubernetes.io/projected/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-kube-api-access-88qmw\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.526659 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526657    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.526659 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526666    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.526860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526674    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.526860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526682    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.526860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.526690    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:41:19.590636 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.590609    2556 generic.go:356] "Generic (PLEG): container finished" podID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" containerID="ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c" exitCode=0
Jul 14 18:41:19.590762 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.590646    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" event={"ID":"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b","Type":"ContainerDied","Data":"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"}
Jul 14 18:41:19.590762 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.590674    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r" event={"ID":"bfad2db2-f354-40a0-b2ef-5dc9d6426b4b","Type":"ContainerDied","Data":"58e59414f15fc82992c60f3ed230fe748793fb1dc1d7eab0de2cde8f9b85afe4"}
Jul 14 18:41:19.590762 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.590692    2556 scope.go:117] "RemoveContainer" containerID="ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"
Jul 14 18:41:19.590762 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.590700    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"
Jul 14 18:41:19.599048 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.599023    2556 scope.go:117] "RemoveContainer" containerID="ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"
Jul 14 18:41:19.599312 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:41:19.599294    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c\": container with ID starting with ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c not found: ID does not exist" containerID="ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"
Jul 14 18:41:19.599367 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.599319    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c"} err="failed to get container status \"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c\": rpc error: code = NotFound desc = could not find container \"ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c\": container with ID starting with ddeb279cf22ee0c4255751f355fca925df875fd44e7acee42606533672e10f9c not found: ID does not exist"
Jul 14 18:41:19.611201 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.611167    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:41:19.617127 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:19.617104    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/auth-test-model-b860a5-kserve-75d749575f-gtt9r"]
Jul 14 18:41:20.319393 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:20.319360    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" path="/var/lib/kubelet/pods/985b6b3a-bd50-4990-bb9c-0340d3cb0040/volumes"
Jul 14 18:41:20.319745 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:20.319730    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" path="/var/lib/kubelet/pods/bfad2db2-f354-40a0-b2ef-5dc9d6426b4b/volumes"
Jul 14 18:41:33.679822 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.679793    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:41:33.680184 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680133    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" containerName="main"
Jul 14 18:41:33.680184 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680149    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" containerName="main"
Jul 14 18:41:33.680184 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680162    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" containerName="main"
Jul 14 18:41:33.680184 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680167    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" containerName="main"
Jul 14 18:41:33.680312 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680234    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfad2db2-f354-40a0-b2ef-5dc9d6426b4b" containerName="main"
Jul 14 18:41:33.680312 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.680244    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="985b6b3a-bd50-4990-bb9c-0340d3cb0040" containerName="main"
Jul 14 18:41:33.684374 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.684357    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.687730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.687696    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"default-dockercfg-46zf8\""
Jul 14 18:41:33.687730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.687703    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"kube-root-ca.crt\""
Jul 14 18:41:33.687891 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.687735    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"openshift-service-ca.crt\""
Jul 14 18:41:33.687891 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.687802    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"sub-test-model-b860a5-kserve-self-signed-certs\""
Jul 14 18:41:33.699051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.699030    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:41:33.737726 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.737851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737741    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzg9q\" (UniqueName: \"kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.737851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737769    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.737851 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737836    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.737997 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737876    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.737997 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.737901    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.838849 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.838817    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.838986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.838858    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzg9q\" (UniqueName: \"kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.838986 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.838876    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839099 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.838998    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839099 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.839059    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839099 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.839086    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839242 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.839198    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839335 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.839309    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.839429 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.839362    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.841256 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.841239    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.841509 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.841493    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.847607 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.847562    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzg9q\" (UniqueName: \"kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q\") pod \"sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") " pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:33.993058 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:33.992987    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:34.122652 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:34.122624    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:41:34.125145 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:41:34.125117    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e397c22_2a18_46cf_8f96_c5d340ddad7f.slice/crio-e83b349fb16b39721203a085cb04db6071feb94be5df882e2769124f583b85cb WatchSource:0}: Error finding container e83b349fb16b39721203a085cb04db6071feb94be5df882e2769124f583b85cb: Status 404 returned error can't find the container with id e83b349fb16b39721203a085cb04db6071feb94be5df882e2769124f583b85cb
Jul 14 18:41:34.660639 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:34.660604    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" event={"ID":"0e397c22-2a18-46cf-8f96-c5d340ddad7f","Type":"ContainerStarted","Data":"d03b822032d3758df95337e92d0e06e8e7b921ab9809ddc013025c82adebf705"}
Jul 14 18:41:34.660897 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:34.660648    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" event={"ID":"0e397c22-2a18-46cf-8f96-c5d340ddad7f","Type":"ContainerStarted","Data":"e83b349fb16b39721203a085cb04db6071feb94be5df882e2769124f583b85cb"}
Jul 14 18:41:34.695933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:34.695887    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" podStartSLOduration=1.695870913 podStartE2EDuration="1.695870913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:41:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:41:34.694993698 +0000 UTC m=+2812.896548836" watchObservedRunningTime="2026-07-14 18:41:34.695870913 +0000 UTC m=+2812.897426053"
Jul 14 18:41:43.993684 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:43.993641    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:43.993684 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:43.993689    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:44.005677 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:44.005647    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:41:44.714813 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:41:44.714790    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:42:21.770279 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:21.770236    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:42:36.789858 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:36.789758    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" podUID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" containerName="main" containerID="cri-o://d03b822032d3758df95337e92d0e06e8e7b921ab9809ddc013025c82adebf705" gracePeriod=45
Jul 14 18:42:36.942823 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:36.942778    2556 generic.go:356] "Generic (PLEG): container finished" podID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" containerID="d03b822032d3758df95337e92d0e06e8e7b921ab9809ddc013025c82adebf705" exitCode=0
Jul 14 18:42:36.942968 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:36.942844    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" event={"ID":"0e397c22-2a18-46cf-8f96-c5d340ddad7f","Type":"ContainerDied","Data":"d03b822032d3758df95337e92d0e06e8e7b921ab9809ddc013025c82adebf705"}
Jul 14 18:42:37.037943 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.037918    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:42:37.124526 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124500    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.124692 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124544    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzg9q\" (UniqueName: \"kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.124692 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124571    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.124817 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124708    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.124817 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124754    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.124929 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124812    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:42:37.124929 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124830    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir\") pod \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\" (UID: \"0e397c22-2a18-46cf-8f96-c5d340ddad7f\") "
Jul 14 18:42:37.125007 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.124941    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home" (OuterVolumeSpecName: "home") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:42:37.125050 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.125021    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:42:37.125195 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.125174    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.125278 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.125203    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.125387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.125341    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.126722 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.126698    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:42:37.126801 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.126751    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm" (OuterVolumeSpecName: "dshm") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:42:37.126801 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.126773    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q" (OuterVolumeSpecName: "kube-api-access-mzg9q") pod "0e397c22-2a18-46cf-8f96-c5d340ddad7f" (UID: "0e397c22-2a18-46cf-8f96-c5d340ddad7f"). InnerVolumeSpecName "kube-api-access-mzg9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:42:37.226065 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.226046    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e397c22-2a18-46cf-8f96-c5d340ddad7f-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.226065 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.226065    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e397c22-2a18-46cf-8f96-c5d340ddad7f-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.226204 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.226076    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzg9q\" (UniqueName: \"kubernetes.io/projected/0e397c22-2a18-46cf-8f96-c5d340ddad7f-kube-api-access-mzg9q\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:42:37.948319 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.948287    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"
Jul 14 18:42:37.948319 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.948294    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt" event={"ID":"0e397c22-2a18-46cf-8f96-c5d340ddad7f","Type":"ContainerDied","Data":"e83b349fb16b39721203a085cb04db6071feb94be5df882e2769124f583b85cb"}
Jul 14 18:42:37.948879 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.948350    2556 scope.go:117] "RemoveContainer" containerID="d03b822032d3758df95337e92d0e06e8e7b921ab9809ddc013025c82adebf705"
Jul 14 18:42:37.974842 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.974816    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:42:37.981730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:37.981705    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/sub-test-model-b860a5-kserve-5bf56ffbb7-vhqnt"]
Jul 14 18:42:38.319410 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:42:38.319339    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" path="/var/lib/kubelet/pods/0e397c22-2a18-46cf-8f96-c5d340ddad7f/volumes"
Jul 14 18:43:58.980239 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.980207    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:43:58.980722 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.980497    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" containerName="main"
Jul 14 18:43:58.980722 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.980507    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" containerName="main"
Jul 14 18:43:58.980722 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.980564    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e397c22-2a18-46cf-8f96-c5d340ddad7f" containerName="main"
Jul 14 18:43:58.983355 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.983337    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:58.986634 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.986614    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"kube-root-ca.crt\""
Jul 14 18:43:58.986795 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.986771    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"openshift-service-ca.crt\""
Jul 14 18:43:58.987389 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.987363    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"default-dockercfg-46zf8\""
Jul 14 18:43:58.987470 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.987370    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b860a5\"/\"rate-test-model-b860a5-kserve-self-signed-certs\""
Jul 14 18:43:58.995168 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:58.995146    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:43:59.031952 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.031928    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.032051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.031956    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.032051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.031976    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.032051 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.032005    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.032205 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.032066    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-459m5\" (UniqueName: \"kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.032205 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.032113    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.132971 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.132945    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133069 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.132973    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-459m5\" (UniqueName: \"kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133069 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.132994    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133069 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133048    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133244 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133072    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133244 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133101    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133387 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133368    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133441 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133392    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.133479 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.133465    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.135215 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.135190    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.135570 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.135551    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.144803 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.144783    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-459m5\" (UniqueName: \"kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5\") pod \"rate-test-model-b860a5-kserve-584d97b96c-npcnd\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") " pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.292139 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.292082    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:43:59.418785 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:43:59.418761    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:43:59.421426 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:43:59.421399    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05662f48_4a08_4382_92d2_07dc31f2edf7.slice/crio-2b80f37d0dd0732bdaf27ce89e55674857e88a2e9bf2bee3b3f71675a07dbba9 WatchSource:0}: Error finding container 2b80f37d0dd0732bdaf27ce89e55674857e88a2e9bf2bee3b3f71675a07dbba9: Status 404 returned error can't find the container with id 2b80f37d0dd0732bdaf27ce89e55674857e88a2e9bf2bee3b3f71675a07dbba9
Jul 14 18:44:00.290687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:00.290652    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" event={"ID":"05662f48-4a08-4382-92d2-07dc31f2edf7","Type":"ContainerStarted","Data":"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"}
Jul 14 18:44:00.290687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:00.290699    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" event={"ID":"05662f48-4a08-4382-92d2-07dc31f2edf7","Type":"ContainerStarted","Data":"2b80f37d0dd0732bdaf27ce89e55674857e88a2e9bf2bee3b3f71675a07dbba9"}
Jul 14 18:44:00.319606 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:00.319515    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" podStartSLOduration=2.319503776 podStartE2EDuration="2.319503776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:43:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:44:00.319075282 +0000 UTC m=+2958.520630424" watchObservedRunningTime="2026-07-14 18:44:00.319503776 +0000 UTC m=+2958.521058914"
Jul 14 18:44:09.292422 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:09.292336    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:44:09.292422 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:09.292392    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:44:09.304327 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:09.304300    2556 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:44:09.340258 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:09.340237    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:44:42.365883 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:42.365856    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:44:42.372424 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:42.372400    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:44:42.374147 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:42.374127    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:44:42.380351 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:42.380334    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-d4dwf_3e5d7f17-9f56-405b-a9fb-3140a1504546/ovn-acl-logging/0.log"
Jul 14 18:44:47.476231 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:47.476196    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:44:51.749394 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.749357    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320126891184"]
Jul 14 18:44:51.752574 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.752557    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:51.755496 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.755474    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-lddj2\""
Jul 14 18:44:51.761627 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.761606    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320126891184"]
Jul 14 18:44:51.854448 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.854418    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9scs\" (UniqueName: \"kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs\") pod \"test-curl-7546-140320126891184\" (UID: \"905db159-7b39-4cf0-bb13-2a2ba8b9a731\") " pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:51.955421 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.955387    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9scs\" (UniqueName: \"kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs\") pod \"test-curl-7546-140320126891184\" (UID: \"905db159-7b39-4cf0-bb13-2a2ba8b9a731\") " pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:51.964354 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:51.964329    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9scs\" (UniqueName: \"kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs\") pod \"test-curl-7546-140320126891184\" (UID: \"905db159-7b39-4cf0-bb13-2a2ba8b9a731\") " pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:52.061994 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:52.061908    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:52.192928 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:52.192901    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320126891184"]
Jul 14 18:44:52.195298 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:44:52.195270    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod905db159_7b39_4cf0_bb13_2a2ba8b9a731.slice/crio-e80e408e911f3a350bcb1b6346fbf8ba984aa50d25a0428e854f015d7240ec2f WatchSource:0}: Error finding container e80e408e911f3a350bcb1b6346fbf8ba984aa50d25a0428e854f015d7240ec2f: Status 404 returned error can't find the container with id e80e408e911f3a350bcb1b6346fbf8ba984aa50d25a0428e854f015d7240ec2f
Jul 14 18:44:52.196604 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:52.196570    2556 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 14 18:44:52.511642 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:52.511605    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320126891184" event={"ID":"905db159-7b39-4cf0-bb13-2a2ba8b9a731","Type":"ContainerStarted","Data":"e80e408e911f3a350bcb1b6346fbf8ba984aa50d25a0428e854f015d7240ec2f"}
Jul 14 18:44:53.516752 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:53.516707    2556 generic.go:356] "Generic (PLEG): container finished" podID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" containerID="5cce772c9e8788d86c4c9a6ff45734e8048bad603f3fc4849c8ed2f91b0b182f" exitCode=0
Jul 14 18:44:53.517106 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:53.516796    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320126891184" event={"ID":"905db159-7b39-4cf0-bb13-2a2ba8b9a731","Type":"ContainerDied","Data":"5cce772c9e8788d86c4c9a6ff45734e8048bad603f3fc4849c8ed2f91b0b182f"}
Jul 14 18:44:54.656946 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.656918    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:54.673080 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.672912    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l9scs\" (UniqueName: \"kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs\") pod \"905db159-7b39-4cf0-bb13-2a2ba8b9a731\" (UID: \"905db159-7b39-4cf0-bb13-2a2ba8b9a731\") "
Jul 14 18:44:54.675559 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.675529    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs" (OuterVolumeSpecName: "kube-api-access-l9scs") pod "905db159-7b39-4cf0-bb13-2a2ba8b9a731" (UID: "905db159-7b39-4cf0-bb13-2a2ba8b9a731"). InnerVolumeSpecName "kube-api-access-l9scs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:44:54.701801 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.701768    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320126891184_905db159-7b39-4cf0-bb13-2a2ba8b9a731/test-curl-7546-140320126891184/0.log"
Jul 14 18:44:54.726791 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.726757    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320126891184"]
Jul 14 18:44:54.728750 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.728730    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320126891184"]
Jul 14 18:44:54.773959 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:54.773906    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l9scs\" (UniqueName: \"kubernetes.io/projected/905db159-7b39-4cf0-bb13-2a2ba8b9a731-kube-api-access-l9scs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:44:55.527158 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:55.527127    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:55.527315 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:55.527180    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e80e408e911f3a350bcb1b6346fbf8ba984aa50d25a0428e854f015d7240ec2f"
Jul 14 18:44:55.536628 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:44:55.536605    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7546-140320126891184\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" podUID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" pod="openshift-ingress/test-curl-7546-140320126891184"
Jul 14 18:44:56.319340 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:56.319308    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" path="/var/lib/kubelet/pods/905db159-7b39-4cf0-bb13-2a2ba8b9a731/volumes"
Jul 14 18:44:57.792544 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.792508    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"]
Jul 14 18:44:57.793015 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.792996    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" containerName="test-curl-7546-140320126891184"
Jul 14 18:44:57.793088 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.793018    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" containerName="test-curl-7546-140320126891184"
Jul 14 18:44:57.793142 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.793128    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="905db159-7b39-4cf0-bb13-2a2ba8b9a731" containerName="test-curl-7546-140320126891184"
Jul 14 18:44:57.796262 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.796242    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:44:57.799164 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.799148    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-lddj2\""
Jul 14 18:44:57.807168 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.807150    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"]
Jul 14 18:44:57.895903 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.895880    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vdns\" (UniqueName: \"kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns\") pod \"test-curl-7546-140320127037488\" (UID: \"73038246-5900-48b3-9f1c-a38df0293cc6\") " pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:44:57.996833 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:57.996804    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vdns\" (UniqueName: \"kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns\") pod \"test-curl-7546-140320127037488\" (UID: \"73038246-5900-48b3-9f1c-a38df0293cc6\") " pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:44:58.006990 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:58.006964    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vdns\" (UniqueName: \"kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns\") pod \"test-curl-7546-140320127037488\" (UID: \"73038246-5900-48b3-9f1c-a38df0293cc6\") " pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:44:58.104743 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:58.104709    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:44:58.233073 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:58.232945    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"]
Jul 14 18:44:58.235963 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:44:58.235934    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73038246_5900_48b3_9f1c_a38df0293cc6.slice/crio-d4926f9c95b09e4c6e310abac7e207671186d792ecd1d16ac0123828c9c28899 WatchSource:0}: Error finding container d4926f9c95b09e4c6e310abac7e207671186d792ecd1d16ac0123828c9c28899: Status 404 returned error can't find the container with id d4926f9c95b09e4c6e310abac7e207671186d792ecd1d16ac0123828c9c28899
Jul 14 18:44:58.540976 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:58.540899    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127037488" event={"ID":"73038246-5900-48b3-9f1c-a38df0293cc6","Type":"ContainerStarted","Data":"d4926f9c95b09e4c6e310abac7e207671186d792ecd1d16ac0123828c9c28899"}
Jul 14 18:44:59.547471 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:59.547435    2556 generic.go:356] "Generic (PLEG): container finished" podID="73038246-5900-48b3-9f1c-a38df0293cc6" containerID="2e1d770c0c922fb5c48c80aa3ba463fe03faa488f713c4de272d8725afb2c079" exitCode=0
Jul 14 18:44:59.547848 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:44:59.547524    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127037488" event={"ID":"73038246-5900-48b3-9f1c-a38df0293cc6","Type":"ContainerDied","Data":"2e1d770c0c922fb5c48c80aa3ba463fe03faa488f713c4de272d8725afb2c079"}
Jul 14 18:45:00.674254 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.674229    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:45:00.701199 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.701177    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320127037488_73038246-5900-48b3-9f1c-a38df0293cc6/test-curl-7546-140320127037488/0.log"
Jul 14 18:45:00.718389 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.718368    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vdns\" (UniqueName: \"kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns\") pod \"73038246-5900-48b3-9f1c-a38df0293cc6\" (UID: \"73038246-5900-48b3-9f1c-a38df0293cc6\") "
Jul 14 18:45:00.720284 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.720261    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns" (OuterVolumeSpecName: "kube-api-access-4vdns") pod "73038246-5900-48b3-9f1c-a38df0293cc6" (UID: "73038246-5900-48b3-9f1c-a38df0293cc6"). InnerVolumeSpecName "kube-api-access-4vdns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:00.732360 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.732337    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"]
Jul 14 18:45:00.740160 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.740138    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320127037488"]
Jul 14 18:45:00.818915 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.818864    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vdns\" (UniqueName: \"kubernetes.io/projected/73038246-5900-48b3-9f1c-a38df0293cc6-kube-api-access-4vdns\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:00.865623 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.865598    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:00.865919 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.865906    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" containerName="test-curl-7546-140320127037488"
Jul 14 18:45:00.865981 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.865920    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" containerName="test-curl-7546-140320127037488"
Jul 14 18:45:00.866018 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.865982    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" containerName="test-curl-7546-140320127037488"
Jul 14 18:45:00.868659 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.868645    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:00.876932 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.876914    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:00.919432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:00.919403    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xczwn\" (UniqueName: \"kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn\") pod \"test-curl-7546-140320127038496\" (UID: \"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:01.019845 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.019821    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xczwn\" (UniqueName: \"kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn\") pod \"test-curl-7546-140320127038496\" (UID: \"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:01.028528 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.028502    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xczwn\" (UniqueName: \"kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn\") pod \"test-curl-7546-140320127038496\" (UID: \"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:01.177464 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.177444    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:01.300336 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.300311    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:01.302435 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:45:01.302410    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad007e1f_c4ce_47f1_a065_e73b9a4d40b8.slice/crio-181d49b435752063b4f077b4aba76a118591bf0e125d86540539654a9a17703a WatchSource:0}: Error finding container 181d49b435752063b4f077b4aba76a118591bf0e125d86540539654a9a17703a: Status 404 returned error can't find the container with id 181d49b435752063b4f077b4aba76a118591bf0e125d86540539654a9a17703a
Jul 14 18:45:01.557188 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.557112    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127038496" event={"ID":"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8","Type":"ContainerStarted","Data":"181d49b435752063b4f077b4aba76a118591bf0e125d86540539654a9a17703a"}
Jul 14 18:45:01.558283 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.558255    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d4926f9c95b09e4c6e310abac7e207671186d792ecd1d16ac0123828c9c28899"
Jul 14 18:45:01.558283 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:01.558283    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:45:01.570654 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:01.570625    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7546-140320127037488\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:45:02.317715 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:02.317662    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7546-140320127037488\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" pod="openshift-ingress/test-curl-7546-140320127037488"
Jul 14 18:45:02.319391 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.319365    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="73038246-5900-48b3-9f1c-a38df0293cc6" path="/var/lib/kubelet/pods/73038246-5900-48b3-9f1c-a38df0293cc6/volumes"
Jul 14 18:45:02.496784 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.496746    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" podUID="05662f48-4a08-4382-92d2-07dc31f2edf7" containerName="main" containerID="cri-o://cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7" gracePeriod=45
Jul 14 18:45:02.564486 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.564454    2556 generic.go:356] "Generic (PLEG): container finished" podID="ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" containerID="2eed4a6a8253397c321581b1341e4253932b27f51578d02946f44edbcbbb92c3" exitCode=0
Jul 14 18:45:02.564645 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.564543    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127038496" event={"ID":"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8","Type":"ContainerDied","Data":"2eed4a6a8253397c321581b1341e4253932b27f51578d02946f44edbcbbb92c3"}
Jul 14 18:45:02.738556 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.738535    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:45:02.834625 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834577    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-459m5\" (UniqueName: \"kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834645    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834671    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834693    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834708    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834748 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834731    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache\") pod \"05662f48-4a08-4382-92d2-07dc31f2edf7\" (UID: \"05662f48-4a08-4382-92d2-07dc31f2edf7\") "
Jul 14 18:45:02.834979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834947    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:45:02.834979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.834958    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home" (OuterVolumeSpecName: "home") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:45:02.835102 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.835080    2556 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-tmp-dir\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:02.835102 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.835075    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache" (OuterVolumeSpecName: "model-cache") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:45:02.836737 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.836712    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5" (OuterVolumeSpecName: "kube-api-access-459m5") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "kube-api-access-459m5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:02.836820 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.836778    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm" (OuterVolumeSpecName: "dshm") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 14 18:45:02.836867 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.836836    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "05662f48-4a08-4382-92d2-07dc31f2edf7" (UID: "05662f48-4a08-4382-92d2-07dc31f2edf7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:45:02.936252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.936189    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-459m5\" (UniqueName: \"kubernetes.io/projected/05662f48-4a08-4382-92d2-07dc31f2edf7-kube-api-access-459m5\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:02.936252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.936211    2556 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-home\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:02.936252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.936222    2556 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/05662f48-4a08-4382-92d2-07dc31f2edf7-tls-certs\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:02.936252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.936231    2556 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-dshm\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:02.936252 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:02.936239    2556 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/05662f48-4a08-4382-92d2-07dc31f2edf7-model-cache\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:03.570314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.570276    2556 generic.go:356] "Generic (PLEG): container finished" podID="05662f48-4a08-4382-92d2-07dc31f2edf7" containerID="cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7" exitCode=0
Jul 14 18:45:03.570314 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.570312    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"
Jul 14 18:45:03.570809 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.570364    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" event={"ID":"05662f48-4a08-4382-92d2-07dc31f2edf7","Type":"ContainerDied","Data":"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"}
Jul 14 18:45:03.570809 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.570400    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd" event={"ID":"05662f48-4a08-4382-92d2-07dc31f2edf7","Type":"ContainerDied","Data":"2b80f37d0dd0732bdaf27ce89e55674857e88a2e9bf2bee3b3f71675a07dbba9"}
Jul 14 18:45:03.570809 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.570420    2556 scope.go:117] "RemoveContainer" containerID="cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"
Jul 14 18:45:03.580270 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.580255    2556 scope.go:117] "RemoveContainer" containerID="cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"
Jul 14 18:45:03.580513 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:03.580496    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7\": container with ID starting with cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7 not found: ID does not exist" containerID="cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"
Jul 14 18:45:03.580567 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.580520    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7"} err="failed to get container status \"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7\": rpc error: code = NotFound desc = could not find container \"cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7\": container with ID starting with cb948c7a0818ead77b1c33d5c250399a403617586dddb6dfdd0b628f614c53b7 not found: ID does not exist"
Jul 14 18:45:03.592960 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.592937    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:45:03.596802 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.596780    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b860a5/rate-test-model-b860a5-kserve-584d97b96c-npcnd"]
Jul 14 18:45:03.693153 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.693133    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:03.718008 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.717985    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320127038496_ad007e1f-c4ce-47f1-a065-e73b9a4d40b8/test-curl-7546-140320127038496/0.log"
Jul 14 18:45:03.742241 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.742217    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xczwn\" (UniqueName: \"kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn\") pod \"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8\" (UID: \"ad007e1f-c4ce-47f1-a065-e73b9a4d40b8\") "
Jul 14 18:45:03.744212 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.744191    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn" (OuterVolumeSpecName: "kube-api-access-xczwn") pod "ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" (UID: "ad007e1f-c4ce-47f1-a065-e73b9a4d40b8"). InnerVolumeSpecName "kube-api-access-xczwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:03.747371 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.747317    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:03.749782 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.749759    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:03.843454 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.843400    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xczwn\" (UniqueName: \"kubernetes.io/projected/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8-kube-api-access-xczwn\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:03.895394 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895345    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:03.895745 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895726    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05662f48-4a08-4382-92d2-07dc31f2edf7" containerName="main"
Jul 14 18:45:03.895820 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895749    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="05662f48-4a08-4382-92d2-07dc31f2edf7" containerName="main"
Jul 14 18:45:03.895820 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895763    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:03.895820 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895772    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:03.895926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895874    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="05662f48-4a08-4382-92d2-07dc31f2edf7" containerName="main"
Jul 14 18:45:03.895926 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.895885    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:03.898916 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.898900    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:03.910730 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.910708    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:03.944666 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:03.944625    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-956vk\" (UniqueName: \"kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk\") pod \"test-curl-7546-140320127038496\" (UID: \"e436a3bd-7bac-4f78-ba7e-91871cf057e4\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:04.045086 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.045047    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-956vk\" (UniqueName: \"kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk\") pod \"test-curl-7546-140320127038496\" (UID: \"e436a3bd-7bac-4f78-ba7e-91871cf057e4\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:04.053902 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.053861    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-956vk\" (UniqueName: \"kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk\") pod \"test-curl-7546-140320127038496\" (UID: \"e436a3bd-7bac-4f78-ba7e-91871cf057e4\") " pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:04.207769 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.207730    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:04.320553 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.320520    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05662f48-4a08-4382-92d2-07dc31f2edf7" path="/var/lib/kubelet/pods/05662f48-4a08-4382-92d2-07dc31f2edf7/volumes"
Jul 14 18:45:04.320879 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.320863    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad007e1f-c4ce-47f1-a065-e73b9a4d40b8" path="/var/lib/kubelet/pods/ad007e1f-c4ce-47f1-a065-e73b9a4d40b8/volumes"
Jul 14 18:45:04.344957 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.344918    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:04.347931 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:45:04.347872    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode436a3bd_7bac_4f78_ba7e_91871cf057e4.slice/crio-2fce7e320fd9376bd790d71161f02ed392109a47ad46e0c11826e500a726fc03 WatchSource:0}: Error finding container 2fce7e320fd9376bd790d71161f02ed392109a47ad46e0c11826e500a726fc03: Status 404 returned error can't find the container with id 2fce7e320fd9376bd790d71161f02ed392109a47ad46e0c11826e500a726fc03
Jul 14 18:45:04.575633 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.575560    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:04.575633 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.575593    2556 scope.go:117] "RemoveContainer" containerID="2eed4a6a8253397c321581b1341e4253932b27f51578d02946f44edbcbbb92c3"
Jul 14 18:45:04.576960 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:04.576934    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127038496" event={"ID":"e436a3bd-7bac-4f78-ba7e-91871cf057e4","Type":"ContainerStarted","Data":"2fce7e320fd9376bd790d71161f02ed392109a47ad46e0c11826e500a726fc03"}
Jul 14 18:45:05.584931 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:05.584898    2556 generic.go:356] "Generic (PLEG): container finished" podID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" containerID="457499c1a397a675e776bbc0952bd504af142fa50267db395acf87a819cce7df" exitCode=0
Jul 14 18:45:05.585260 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:05.584962    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7546-140320127038496" event={"ID":"e436a3bd-7bac-4f78-ba7e-91871cf057e4","Type":"ContainerDied","Data":"457499c1a397a675e776bbc0952bd504af142fa50267db395acf87a819cce7df"}
Jul 14 18:45:06.719295 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.719273    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:06.743882 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.743862    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7546-140320127038496_e436a3bd-7bac-4f78-ba7e-91871cf057e4/test-curl-7546-140320127038496/0.log"
Jul 14 18:45:06.767223 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.767196    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:06.767811 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.767795    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-956vk\" (UniqueName: \"kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk\") pod \"e436a3bd-7bac-4f78-ba7e-91871cf057e4\" (UID: \"e436a3bd-7bac-4f78-ba7e-91871cf057e4\") "
Jul 14 18:45:06.770182 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.770149    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk" (OuterVolumeSpecName: "kube-api-access-956vk") pod "e436a3bd-7bac-4f78-ba7e-91871cf057e4" (UID: "e436a3bd-7bac-4f78-ba7e-91871cf057e4"). InnerVolumeSpecName "kube-api-access-956vk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:06.771309 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.771291    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7546-140320127038496"]
Jul 14 18:45:06.868308 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:06.868287    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-956vk\" (UniqueName: \"kubernetes.io/projected/e436a3bd-7bac-4f78-ba7e-91871cf057e4-kube-api-access-956vk\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:07.594305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:07.594277    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fce7e320fd9376bd790d71161f02ed392109a47ad46e0c11826e500a726fc03"
Jul 14 18:45:07.594305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:07.594301    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:07.603622 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:07.603569    2556 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7546-140320127038496\" is forbidden: User \"system:node:ip-10-0-130-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-130-101.ec2.internal' and this object" podUID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" pod="openshift-ingress/test-curl-7546-140320127038496"
Jul 14 18:45:08.318709 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:08.318674    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" path="/var/lib/kubelet/pods/e436a3bd-7bac-4f78-ba7e-91871cf057e4/volumes"
Jul 14 18:45:17.632257 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.632227    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:45:17.632810 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.632473    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" podUID="5b307d03-8f40-4d92-80dc-247326fc0c1e" containerName="maas-api" containerID="cri-o://988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23" gracePeriod=30
Jul 14 18:45:17.877933 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.877911    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:45:17.960960 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.960895    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") pod \"5b307d03-8f40-4d92-80dc-247326fc0c1e\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") "
Jul 14 18:45:17.960960 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.960947    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2qc8\" (UniqueName: \"kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8\") pod \"5b307d03-8f40-4d92-80dc-247326fc0c1e\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") "
Jul 14 18:45:17.961148 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.961012    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca\") pod \"5b307d03-8f40-4d92-80dc-247326fc0c1e\" (UID: \"5b307d03-8f40-4d92-80dc-247326fc0c1e\") "
Jul 14 18:45:17.961391 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.961362    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5b307d03-8f40-4d92-80dc-247326fc0c1e" (UID: "5b307d03-8f40-4d92-80dc-247326fc0c1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:45:17.963105 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.963081    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5b307d03-8f40-4d92-80dc-247326fc0c1e" (UID: "5b307d03-8f40-4d92-80dc-247326fc0c1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:45:17.963202 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:17.963167    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8" (OuterVolumeSpecName: "kube-api-access-k2qc8") pod "5b307d03-8f40-4d92-80dc-247326fc0c1e" (UID: "5b307d03-8f40-4d92-80dc-247326fc0c1e"). InnerVolumeSpecName "kube-api-access-k2qc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:18.061826 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.061795    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b307d03-8f40-4d92-80dc-247326fc0c1e-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:18.061826 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.061821    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5b307d03-8f40-4d92-80dc-247326fc0c1e-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:18.061983 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.061831    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2qc8\" (UniqueName: \"kubernetes.io/projected/5b307d03-8f40-4d92-80dc-247326fc0c1e-kube-api-access-k2qc8\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:18.642047 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.642011    2556 generic.go:356] "Generic (PLEG): container finished" podID="5b307d03-8f40-4d92-80dc-247326fc0c1e" containerID="988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23" exitCode=0
Jul 14 18:45:18.642432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.642056    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"
Jul 14 18:45:18.642432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.642103    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" event={"ID":"5b307d03-8f40-4d92-80dc-247326fc0c1e","Type":"ContainerDied","Data":"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"}
Jul 14 18:45:18.642432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.642147    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq" event={"ID":"5b307d03-8f40-4d92-80dc-247326fc0c1e","Type":"ContainerDied","Data":"71b13b02e87463b18e0de18fc11480c73f2b1a580c1b45e1e1ba013ea9be8141"}
Jul 14 18:45:18.642432 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.642167    2556 scope.go:117] "RemoveContainer" containerID="988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"
Jul 14 18:45:18.651391 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.651370    2556 scope.go:117] "RemoveContainer" containerID="988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"
Jul 14 18:45:18.651678 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:18.651658    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23\": container with ID starting with 988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23 not found: ID does not exist" containerID="988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"
Jul 14 18:45:18.651756 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.651686    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23"} err="failed to get container status \"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23\": rpc error: code = NotFound desc = could not find container \"988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23\": container with ID starting with 988679fc18a1d0a2c6c98d85160ed240877f89ec1b584558daf4e0b88193bf23 not found: ID does not exist"
Jul 14 18:45:18.662165 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.662141    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:45:18.672709 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:18.672690    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-7e7ccb-759dcd685f-xgmlq"]
Jul 14 18:45:20.318346 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:20.318312    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b307d03-8f40-4d92-80dc-247326fc0c1e" path="/var/lib/kubelet/pods/5b307d03-8f40-4d92-80dc-247326fc0c1e/volumes"
Jul 14 18:45:33.217962 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.217887    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"]
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218199    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218211    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218225    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b307d03-8f40-4d92-80dc-247326fc0c1e" containerName="maas-api"
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218230    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b307d03-8f40-4d92-80dc-247326fc0c1e" containerName="maas-api"
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218288    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b307d03-8f40-4d92-80dc-247326fc0c1e" containerName="maas-api"
Jul 14 18:45:33.218305 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.218297    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="e436a3bd-7bac-4f78-ba7e-91871cf057e4" containerName="test-curl-7546-140320127038496"
Jul 14 18:45:33.223019 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.223002    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:33.227008 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.226985    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-pp62l\""
Jul 14 18:45:33.230805 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.230783    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"]
Jul 14 18:45:33.283860 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.283830    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8\" (UID: \"2efb7f84-38a8-4c7a-9231-a48818d9b101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:33.384934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.384900    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8\" (UID: \"2efb7f84-38a8-4c7a-9231-a48818d9b101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:33.385466 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.385449    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8\" (UID: \"2efb7f84-38a8-4c7a-9231-a48818d9b101\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:33.533687 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.533615    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:33.663763 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.663738    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"]
Jul 14 18:45:33.666090 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:45:33.666060    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efb7f84_38a8_4c7a_9231_a48818d9b101.slice/crio-8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f WatchSource:0}: Error finding container 8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f: Status 404 returned error can't find the container with id 8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f
Jul 14 18:45:33.710600 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:33.710553    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8" event={"ID":"2efb7f84-38a8-4c7a-9231-a48818d9b101","Type":"ContainerStarted","Data":"8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f"}
Jul 14 18:45:34.716486 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:34.716452    2556 generic.go:356] "Generic (PLEG): container finished" podID="2efb7f84-38a8-4c7a-9231-a48818d9b101" containerID="457a32fe790468691499c0bc2c0f550360b9a41f743e60a9c20c354bea36bcc6" exitCode=0
Jul 14 18:45:34.716884 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:34.716501    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8" event={"ID":"2efb7f84-38a8-4c7a-9231-a48818d9b101","Type":"ContainerDied","Data":"457a32fe790468691499c0bc2c0f550360b9a41f743e60a9c20c354bea36bcc6"}
Jul 14 18:45:35.848522 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:35.848495    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:36.005836 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.005726    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca\") pod \"2efb7f84-38a8-4c7a-9231-a48818d9b101\" (UID: \"2efb7f84-38a8-4c7a-9231-a48818d9b101\") "
Jul 14 18:45:36.006092 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.006068    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2efb7f84-38a8-4c7a-9231-a48818d9b101" (UID: "2efb7f84-38a8-4c7a-9231-a48818d9b101"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:45:36.106467 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.106417    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2efb7f84-38a8-4c7a-9231-a48818d9b101-maas-api-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:36.726173 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.726134    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8" event={"ID":"2efb7f84-38a8-4c7a-9231-a48818d9b101","Type":"ContainerDied","Data":"8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f"}
Jul 14 18:45:36.726173 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.726166    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8"
Jul 14 18:45:36.726411 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:36.726193    2556 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f05fa60938404b0defece6cde29ba986f53ecc4bac3e2d26c4511011f269a2f"
Jul 14 18:45:43.824782 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:43.824748    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:45:43.825242 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:43.825015    2556 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" containerName="maas-api" containerID="cri-o://1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320" gracePeriod=30
Jul 14 18:45:44.066796 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.066774    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:45:44.168800 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.168774    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7sc89\" (UniqueName: \"kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89\") pod \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") "
Jul 14 18:45:44.168979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.168831    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") pod \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") "
Jul 14 18:45:44.168979 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.168851    2556 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca\") pod \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\" (UID: \"0acddc6d-12ec-4b3a-bcca-08dba795a99b\") "
Jul 14 18:45:44.169292 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.169267    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0acddc6d-12ec-4b3a-bcca-08dba795a99b" (UID: "0acddc6d-12ec-4b3a-bcca-08dba795a99b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 14 18:45:44.171015 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.170994    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89" (OuterVolumeSpecName: "kube-api-access-7sc89") pod "0acddc6d-12ec-4b3a-bcca-08dba795a99b" (UID: "0acddc6d-12ec-4b3a-bcca-08dba795a99b"). InnerVolumeSpecName "kube-api-access-7sc89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 14 18:45:44.171082 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.171033    2556 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0acddc6d-12ec-4b3a-bcca-08dba795a99b" (UID: "0acddc6d-12ec-4b3a-bcca-08dba795a99b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 14 18:45:44.269613 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.269562    2556 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sc89\" (UniqueName: \"kubernetes.io/projected/0acddc6d-12ec-4b3a-bcca-08dba795a99b-kube-api-access-7sc89\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:44.269613 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.269611    2556 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0acddc6d-12ec-4b3a-bcca-08dba795a99b-maas-api-tls\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:44.269774 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.269628    2556 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0acddc6d-12ec-4b3a-bcca-08dba795a99b-openshift-service-ca\") on node \"ip-10-0-130-101.ec2.internal\" DevicePath \"\""
Jul 14 18:45:44.760934 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.760898    2556 generic.go:356] "Generic (PLEG): container finished" podID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" containerID="1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320" exitCode=0
Jul 14 18:45:44.761093 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.760945    2556 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"
Jul 14 18:45:44.761093 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.760983    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" event={"ID":"0acddc6d-12ec-4b3a-bcca-08dba795a99b","Type":"ContainerDied","Data":"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"}
Jul 14 18:45:44.761093 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.761040    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr" event={"ID":"0acddc6d-12ec-4b3a-bcca-08dba795a99b","Type":"ContainerDied","Data":"77e36bb9213a87519ce1c2d8425c9a8ae9d27bc1cc5438690c9f07bb6b84a02e"}
Jul 14 18:45:44.761093 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.761062    2556 scope.go:117] "RemoveContainer" containerID="1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"
Jul 14 18:45:44.769332 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.769309    2556 scope.go:117] "RemoveContainer" containerID="1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"
Jul 14 18:45:44.769639 ip-10-0-130-101 kubenswrapper[2556]: E0714 18:45:44.769617    2556 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320\": container with ID starting with 1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320 not found: ID does not exist" containerID="1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"
Jul 14 18:45:44.769696 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.769649    2556 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320"} err="failed to get container status \"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320\": rpc error: code = NotFound desc = could not find container \"1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320\": container with ID starting with 1f2fcd7c16813c613adb71e9d128556c52c267e7b44fe18e4d13e5e9e804c320 not found: ID does not exist"
Jul 14 18:45:44.778423 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.778401    2556 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:45:44.782199 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:44.782175    2556 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-b860a5-78955d6f5c-dwwqr"]
Jul 14 18:45:46.318882 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:46.318850    2556 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" path="/var/lib/kubelet/pods/0acddc6d-12ec-4b3a-bcca-08dba795a99b/volumes"
Jul 14 18:45:59.806272 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:45:59.806241    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d7d9c478f-znwrw_00b6830b-9392-4d2d-8885-d8638a9b1889/authorino/0.log"
Jul 14 18:46:05.656211 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:05.656176    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-2tr6j_5a0506ca-5185-4654-844a-d3dcc95f4d41/manager/0.log"
Jul 14 18:46:05.663504 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:05.663486    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-2tr6j_5a0506ca-5185-4654-844a-d3dcc95f4d41/copy-manifests/0.log"
Jul 14 18:46:06.858504 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:06.858473    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-3ccf089e-hzngb_2cf214c5-7bee-42c9-a0ed-d5a7f72bba3e/revoke-keys/0.log"
Jul 14 18:46:07.996354 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:07.996326    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-shared-b-b860a5-sppw8_2efb7f84-38a8-4c7a-9231-a48818d9b101/revoke-keys/0.log"
Jul 14 18:46:09.474385 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:09.474359    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d7d9c478f-znwrw_00b6830b-9392-4d2d-8885-d8638a9b1889/authorino/0.log"
Jul 14 18:46:10.091820 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:10.091788    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-m9pm4_e6379105-89d8-49af-ae2a-933e595e39e2/manager/0.log"
Jul 14 18:46:11.054992 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:11.054956    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5979cf877b-pv9n9_0cbd0de3-08a3-407b-b48c-3fa52677c55b/kube-auth-proxy/0.log"
Jul 14 18:46:11.588244 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:11.588201    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6969fbcc8-gndlg_4b333119-42bc-422a-abe2-fdbe41530d5a/router/0.log"
Jul 14 18:46:12.067179 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:12.067128    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gkhj5_e5015c76-f0f4-4b69-8778-491600d05414/main/0.log"
Jul 14 18:46:12.543688 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:12.543663    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-gt8r2_798241e5-e7cb-43b4-8e61-54a6dca46c58/main/0.log"
Jul 14 18:46:20.978185 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:20.978149    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-96rbq_581bd84f-f8fa-4b81-9cdd-067354af5b90/global-pull-secret-syncer/0.log"
Jul 14 18:46:21.200995 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:21.200966    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-p7qc9_e94b10d6-16cc-457b-ad9b-a5fe64b722dc/konnectivity-agent/0.log"
Jul 14 18:46:21.384852 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:21.384824    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-101.ec2.internal_46b36afb4d674d2ff23bc070cd70fd42/haproxy/0.log"
Jul 14 18:46:26.911676 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:26.911632    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7d7d9c478f-znwrw_00b6830b-9392-4d2d-8885-d8638a9b1889/authorino/0.log"
Jul 14 18:46:27.433488 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:27.433456    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-m9pm4_e6379105-89d8-49af-ae2a-933e595e39e2/manager/0.log"
Jul 14 18:46:29.815281 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:29.815226    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fg7dw_b28a5863-4f50-4da9-afc0-e5e3047392f7/node-exporter/0.log"
Jul 14 18:46:29.848677 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:29.848646    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fg7dw_b28a5863-4f50-4da9-afc0-e5e3047392f7/kube-rbac-proxy/0.log"
Jul 14 18:46:29.884260 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:29.884237    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fg7dw_b28a5863-4f50-4da9-afc0-e5e3047392f7/init-textfile/0.log"
Jul 14 18:46:30.175039 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.175015    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/prometheus/0.log"
Jul 14 18:46:30.203768 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.203747    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/config-reloader/0.log"
Jul 14 18:46:30.229888 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.229871    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/thanos-sidecar/0.log"
Jul 14 18:46:30.257941 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.257921    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/kube-rbac-proxy-web/0.log"
Jul 14 18:46:30.290091 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.290067    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/kube-rbac-proxy/0.log"
Jul 14 18:46:30.314857 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.314833    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/kube-rbac-proxy-thanos/0.log"
Jul 14 18:46:30.346012 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:30.345988    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_af77b72e-38e0-4f1b-9fdf-1cf9c4948da3/init-config-reloader/0.log"
Jul 14 18:46:31.745403 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745368    2556 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"]
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745669    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" containerName="maas-api"
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745682    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" containerName="maas-api"
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745702    2556 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2efb7f84-38a8-4c7a-9231-a48818d9b101" containerName="revoke-keys"
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745707    2556 state_mem.go:107] "Deleted CPUSet assignment" podUID="2efb7f84-38a8-4c7a-9231-a48818d9b101" containerName="revoke-keys"
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745755    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="0acddc6d-12ec-4b3a-bcca-08dba795a99b" containerName="maas-api"
Jul 14 18:46:31.745854 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.745762    2556 memory_manager.go:368] "RemoveStaleState removing state" podUID="2efb7f84-38a8-4c7a-9231-a48818d9b101" containerName="revoke-keys"
Jul 14 18:46:31.748617 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.748596    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.752254 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.752227    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qkbmz\"/\"kube-root-ca.crt\""
Jul 14 18:46:31.752391 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.752373    2556 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qkbmz\"/\"openshift-service-ca.crt\""
Jul 14 18:46:31.752449 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.752432    2556 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qkbmz\"/\"default-dockercfg-qfjll\""
Jul 14 18:46:31.759356 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.759334    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"]
Jul 14 18:46:31.827500 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.827464    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-lib-modules\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.827500 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.827499    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpxm5\" (UniqueName: \"kubernetes.io/projected/618e6b00-d2a9-40df-b316-11853cff0585-kube-api-access-jpxm5\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.827759 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.827527    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-sys\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.827759 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.827601    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-podres\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.827759 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.827696    2556 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-proc\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.845522 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.845496    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-jpsxw_d84c77e5-cd9c-4a08-acc2-36a378cba51c/networking-console-plugin/0.log"
Jul 14 18:46:31.928358 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928333    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-proc\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928487 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928377    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-lib-modules\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928487 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928398    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpxm5\" (UniqueName: \"kubernetes.io/projected/618e6b00-d2a9-40df-b316-11853cff0585-kube-api-access-jpxm5\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928487 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928451    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-proc\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928487 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928481    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-sys\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928515    2556 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-podres\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928545    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-lib-modules\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928563    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-sys\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.928700 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.928694    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/618e6b00-d2a9-40df-b316-11853cff0585-podres\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:31.937858 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:31.937835    2556 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpxm5\" (UniqueName: \"kubernetes.io/projected/618e6b00-d2a9-40df-b316-11853cff0585-kube-api-access-jpxm5\") pod \"perf-node-gather-daemonset-6ttqb\" (UID: \"618e6b00-d2a9-40df-b316-11853cff0585\") " pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:32.058278 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.058209    2556 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:32.187217 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.187187    2556 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"]
Jul 14 18:46:32.190688 ip-10-0-130-101 kubenswrapper[2556]: W0714 18:46:32.190651    2556 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod618e6b00_d2a9_40df_b316_11853cff0585.slice/crio-4a49473cf87578b2dbd6aeef641aff789a8dbd0331d36d5d8fc405022a132dd7 WatchSource:0}: Error finding container 4a49473cf87578b2dbd6aeef641aff789a8dbd0331d36d5d8fc405022a132dd7: Status 404 returned error can't find the container with id 4a49473cf87578b2dbd6aeef641aff789a8dbd0331d36d5d8fc405022a132dd7
Jul 14 18:46:32.484438 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.484409    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/1.log"
Jul 14 18:46:32.495814 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.495756    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-s259n_f7e08882-f6ef-4a60-a3a7-cffa01a3452b/console-operator/2.log"
Jul 14 18:46:32.972086 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.972055    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb" event={"ID":"618e6b00-d2a9-40df-b316-11853cff0585","Type":"ContainerStarted","Data":"c6fb58948ecb4e8733fdda7e5cf7f6dee79e4be84deb70f98c0cbe928b566c01"}
Jul 14 18:46:32.972482 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.972110    2556 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb" event={"ID":"618e6b00-d2a9-40df-b316-11853cff0585","Type":"ContainerStarted","Data":"4a49473cf87578b2dbd6aeef641aff789a8dbd0331d36d5d8fc405022a132dd7"}
Jul 14 18:46:32.972482 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.972146    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:32.989249 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:32.989201    2556 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb" podStartSLOduration=1.989187297 podStartE2EDuration="1.989187297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 18:46:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:46:32.988310734 +0000 UTC m=+3111.189865873" watchObservedRunningTime="2026-07-14 18:46:32.989187297 +0000 UTC m=+3111.190742463"
Jul 14 18:46:34.956024 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:34.955997    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4mgqq_a8cef3db-b23e-4da6-817f-40fbc39fd08f/dns/0.log"
Jul 14 18:46:34.985870 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:34.985842    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4mgqq_a8cef3db-b23e-4da6-817f-40fbc39fd08f/kube-rbac-proxy/0.log"
Jul 14 18:46:35.174486 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:35.174462    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2rmp8_3965cd09-acda-428c-bad2-597c0a5b67cf/dns-node-resolver/0.log"
Jul 14 18:46:36.025539 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:36.025505    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b49f4f54f-cq6xc_f8e5fffb-1b22-4802-991b-7036d56a4419/registry/0.log"
Jul 14 18:46:36.132800 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:36.132774    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bs5nf_b0418412-360e-450b-b9e3-89bf9b2632b3/node-ca/0.log"
Jul 14 18:46:37.514205 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:37.514171    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5979cf877b-pv9n9_0cbd0de3-08a3-407b-b48c-3fa52677c55b/kube-auth-proxy/0.log"
Jul 14 18:46:37.866255 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:37.866224    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6969fbcc8-gndlg_4b333119-42bc-422a-abe2-fdbe41530d5a/router/0.log"
Jul 14 18:46:38.784318 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:38.784291    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-kpkc6_d5c2d0a4-f689-468f-856f-280a65f6ec5d/serve-healthcheck-canary/0.log"
Jul 14 18:46:38.986752 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:38.986732    2556 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qkbmz/perf-node-gather-daemonset-6ttqb"
Jul 14 18:46:39.391515 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:39.391492    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m77wx_8dbe2af4-2154-499d-890c-a605845775b5/kube-rbac-proxy/0.log"
Jul 14 18:46:39.420318 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:39.420297    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m77wx_8dbe2af4-2154-499d-890c-a605845775b5/exporter/0.log"
Jul 14 18:46:39.452388 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:39.452369    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m77wx_8dbe2af4-2154-499d-890c-a605845775b5/extractor/0.log"
Jul 14 18:46:41.863050 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:41.863022    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5979cf877b-pv9n9_0cbd0de3-08a3-407b-b48c-3fa52677c55b/kube-auth-proxy/0.log"
Jul 14 18:46:42.032131 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:42.032095    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6969fbcc8-gndlg_4b333119-42bc-422a-abe2-fdbe41530d5a/router/0.log"
Jul 14 18:46:43.211992 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:43.211915    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-2tr6j_5a0506ca-5185-4654-844a-d3dcc95f4d41/manager/0.log"
Jul 14 18:46:43.276294 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:43.276275    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67c464fdbc-2tr6j_5a0506ca-5185-4654-844a-d3dcc95f4d41/copy-manifests/0.log"
Jul 14 18:46:43.573947 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:43.573870    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5979cf877b-pv9n9_0cbd0de3-08a3-407b-b48c-3fa52677c55b/kube-auth-proxy/0.log"
Jul 14 18:46:43.767608 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:43.767570    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6969fbcc8-gndlg_4b333119-42bc-422a-abe2-fdbe41530d5a/router/0.log"
Jul 14 18:46:44.931064 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:44.931031    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-57cf6fdf7d-s7pj2_213ec993-5513-469c-b459-d56d4ae5e7f1/manager/0.log"
Jul 14 18:46:44.983054 ip-10-0-130-101 kubenswrapper[2556]: I0714 18:46:44.982993    2556 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-w9fgk_a2c1a031-1ba7-4e94-aa48-9189390ff2fe/openshift-lws-operator/0.log"