Jul 09 13:58:14.321923 ip-10-0-143-235 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 13:58:14.774549 ip-10-0-143-235 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:58:14.774549 ip-10-0-143-235 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 13:58:14.774549 ip-10-0-143-235 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:58:14.774549 ip-10-0-143-235 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 13:58:14.774549 ip-10-0-143-235 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 13:58:14.776673 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.776584    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 13:58:14.784931 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784901    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.784931 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784927    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.784931 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784932    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.784931 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784935    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.784931 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784938    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784941    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784946    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784949    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784951    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784954    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784956    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784959    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784961    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784964    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784966    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784969    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784973    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784977    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784980    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784983    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784986    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784989    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784991    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784995    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.785138 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.784998    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785001    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785003    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785005    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785008    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785011    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785013    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785015    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785018    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785020    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785023    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785026    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785029    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785032    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785034    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785037    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785039    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785041    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785044    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785046    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.785683 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785049    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785052    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785055    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785057    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785060    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785062    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785065    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785067    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785070    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785073    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785075    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785078    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785081    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785083    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785085    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785088    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785090    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785093    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785095    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785097    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.786165 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785099    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785102    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785105    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785108    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785110    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785113    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785116    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785118    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785120    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785123    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785125    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785128    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785130    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785133    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785135    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785138    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785140    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785142    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785145    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785150    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.786687 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785153    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785155    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785158    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785160    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785164    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785166    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785169    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785171    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785174    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785178    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785181    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785183    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785186    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785189    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785192    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785963    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785970    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785973    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785976    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.787170 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785979    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785981    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785984    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785986    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785989    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785991    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785994    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785996    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.785999    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786001    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786004    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786006    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786010    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786015    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786018    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786020    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786023    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786027    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786030    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.787635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786032    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786035    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786038    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786040    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786043    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786045    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786047    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786050    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786052    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786055    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786058    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786060    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786063    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786065    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786068    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786070    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786073    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786075    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786078    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786080    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.788151 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786082    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786085    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786088    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786090    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786093    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786095    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786097    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786100    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786102    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786105    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786108    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786111    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786113    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786115    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786118    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786121    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786124    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786126    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786128    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.788664 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786131    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786133    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786135    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786138    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786140    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786143    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786145    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786148    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786150    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786153    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786155    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786157    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786159    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786162    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786164    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786167    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786170    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786172    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786175    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786177    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.789128 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786180    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786183    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786185    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786188    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786193    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786195    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786198    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786200    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786203    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786205    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786208    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786211    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786215    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786219    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786223    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786227    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786232    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786720    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786726    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.789635 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786730    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786733    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786736    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786739    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786742    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786744    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786747    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786750    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786752    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786755    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786758    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786760    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786763    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786765    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786769    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786771    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786774    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786777    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786779    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786782    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.790095 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786784    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786787    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786789    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786793    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786797    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786800    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786802    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786805    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786807    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786810    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786813    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786815    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786818    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786820    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786822    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786825    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786827    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786829    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786832    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.790603 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786834    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786836    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786839    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786841    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786844    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786846    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786848    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786851    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786854    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786857    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786859    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786862    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786864    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786867    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786869    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786871    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786876    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786879    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786882    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786885    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.791068 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786888    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786891    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786893    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786896    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786898    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786901    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786903    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786906    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786909    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786911    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786914    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786916    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786918    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786921    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786923    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786926    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786928    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786930    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786933    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786935    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.791551 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786937    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786941    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786943    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786946    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786948    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786952    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786955    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786957    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786960    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786963    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786965    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786967    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786970    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786973    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786975    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786977    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786980    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.786982    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787060    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787068    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 13:58:14.792049 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787077    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787084    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787090    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787094    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787101    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787106    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787109    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787112    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787115    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787118    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787121    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787124    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787127    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787130    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787134    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787137    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787142    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787144    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787147    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787150    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787154    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787157    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787163    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 13:58:14.792533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787166    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787170    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787173    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787176    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787179    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787182    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787185    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787189    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787192    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787195    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787198    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787201    2580 flags.go:64] FLAG: --enable-server="true"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787203    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787208    2580 flags.go:64] FLAG: --event-burst="100"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787211    2580 flags.go:64] FLAG: --event-qps="50"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787214    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787217    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787220    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787224    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787228    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787233    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787238    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787243    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787247    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787250    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 13:58:14.793103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787253    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787256    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787259    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787261    2580 flags.go:64] FLAG: --feature-gates=""
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787265    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787271    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787275    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787279    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787282    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787285    2580 flags.go:64] FLAG: --help="false"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787288    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-143-235.ec2.internal"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787291    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787294    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787296    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787300    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787304    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787307    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787309    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787312    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787315    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787318    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787321    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787324    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787327    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 13:58:14.793835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787329    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787332    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787335    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787338    2580 flags.go:64] FLAG: --lock-file=""
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787340    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787343    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787346    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787351    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787355    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787358    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787361    2580 flags.go:64] FLAG: --logging-format="text"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787363    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787366    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787370    2580 flags.go:64] FLAG: --manifest-url=""
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787374    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787379    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787381    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787386    2580 flags.go:64] FLAG: --max-pods="110"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787389    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787392    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787395    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787397    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787400    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787403    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787406    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 13:58:14.794421 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787414    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787417    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787420    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787423    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787426    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787432    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787435    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787438    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787441    2580 flags.go:64] FLAG: --port="10250"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787444    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787448    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02cb8238f8edfe125"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787451    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787454    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787456    2580 flags.go:64] FLAG: --register-node="true"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787459    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787463    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787467    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787469    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787472    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787476    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787479    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787484    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787487    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787490    2580 flags.go:64] FLAG: --runonce="false"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787492    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 13:58:14.795058 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787495    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787498    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787501    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787504    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787507    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787510    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787513    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787516    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787518    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787522    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787524    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787527    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787530    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787533    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787538    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787540    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787543    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787547    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787550    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787553    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787555    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787558    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787561    2580 flags.go:64] FLAG: --v="2"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787578    2580 flags.go:64] FLAG: --version="false"
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787585    2580 flags.go:64] FLAG: --vmodule=""
Jul 09 13:58:14.795690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787589    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.787592    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787720    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787728    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787731    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787734    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787737    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787740    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787743    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787745    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787748    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787750    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787753    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787755    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787758    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787760    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787763    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787765    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787767    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787770    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.796401 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787773    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787775    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787778    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787780    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787782    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787785    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787788    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787790    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787792    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787795    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787798    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787800    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787803    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787805    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787808    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787812    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787814    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787817    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787819    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787822    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.796908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787824    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787827    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787829    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787831    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787834    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787836    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787839    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787841    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787844    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787846    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787849    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787851    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787854    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787856    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787859    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787861    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787863    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787866    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787868    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787871    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.797443 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787873    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787875    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787878    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787880    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787884    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787888    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787891    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787896    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787900    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787903    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787905    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787908    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787910    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787913    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787915    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787917    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787920    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787922    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787925    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.797935 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787927    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787930    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787932    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787934    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787937    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787939    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787942    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787944    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787946    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787949    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787951    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787953    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787956    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787958    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787960    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787963    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787965    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787967    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787970    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787973    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.798403 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787977    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.787979    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.788665    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.796211    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.796234    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.796256    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.796264    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796323    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796328    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796332    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796334    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796337    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796340    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796342    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796345    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.798908 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796347    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796350    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796353    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796356    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796358    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796361    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796364    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796367    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796371    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796375    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796378    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796381    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796383    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796386    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796388    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796390    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796393    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796396    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796398    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796401    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.799286 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796404    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796408    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796411    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796413    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796416    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796419    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796421    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796424    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796426    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796429    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796431    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796433    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796436    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796439    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796441    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796444    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796446    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796448    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796451    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796453    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.799786 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796456    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796461    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796464    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796467    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796470    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796473    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796475    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796478    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796480    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796483    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796485    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796488    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796491    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796494    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796497    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796499    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796501    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796504    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796506    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.800284 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796510    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796513    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796516    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796519    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796522    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796524    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796527    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796529    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796532    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796534    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796536    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796539    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796541    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796544    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796547    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796549    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796552    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796554    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796557    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.800798 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796559    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796562    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796586    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796589    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796591    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796594    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796597    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796600    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796603    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796613    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796615    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796618    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796621    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.796626    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796745    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796749    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 13:58:14.801261 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796752    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796754    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796757    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796760    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796763    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796766    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796769    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796772    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796774    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796777    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796780    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796784    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796787    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796790    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796793    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796796    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796799    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796801    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796804    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796807    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 13:58:14.801720 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796809    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796812    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796815    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796818    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796820    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796823    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796826    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796828    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796830    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796833    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796835    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796838    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796841    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796844    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796847    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796851    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796854    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796857    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796860    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 13:58:14.802203 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796862    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796865    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796867    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796869    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796872    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796875    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796877    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796880    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796882    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796886    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796888    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796891    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796893    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796895    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796897    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796900    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796903    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796906    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796908    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 13:58:14.802685 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796911    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796914    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796917    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796919    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796922    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796924    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796927    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796929    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796931    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796934    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796936    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796938    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796941    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796943    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796946    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796948    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796950    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796952    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796955    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796958    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 13:58:14.803152 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796960    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796963    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796965    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796968    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796970    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796972    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796975    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796977    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796980    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796983    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796985    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796988    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796990    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796992    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796995    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.796998    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.797000    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.797003    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 13:58:14.803645 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:14.797005    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.797010    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.797142    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.799763    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.801530    2580 server.go:1019] "Starting client certificate rotation"
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.801640    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 13:58:14.804074 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.801684    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 13:58:14.804306 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.804289    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 13:58:14.807477 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.807463    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 13:58:14.818222 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.818198    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 09 13:58:14.820791 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.820773    2580 log.go:25] "Validated CRI v1 image API"
Jul 09 13:58:14.821674 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.821654    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 13:58:14.824289 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.824269    2580 fs.go:135] Filesystem UUIDs: map[6eefe09a-e40a-4c1d-bdf1-70d05cda2f89:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 8b9e5f1b-1eca-4f84-9b3b-03267200c316:/dev/nvme0n1p3]
Jul 09 13:58:14.824342 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.824289    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 13:58:14.830342 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.830229    2580 manager.go:216] Machine: {Timestamp:2026-07-09 13:58:14.828770902 +0000 UTC m=+0.370037697 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107309 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2833861fe967a0b157b8e1c503b432 SystemUUID:ec283386-1fe9-67a0-b157-b8e1c503b432 BootID:e22987ac-3e6c-406a-9f91-ed3db30677f3 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c9:9d:58:a8:67 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c9:9d:58:a8:67 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:5a:76:92:dd:2d:9f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 13:58:14.830342 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.830335    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 13:58:14.830464 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.830421    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 13:58:14.833422 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.833404    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 13:58:14.834322 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.834298    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 13:58:14.834459 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.834326    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-235.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 13:58:14.835816 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.835805    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 13:58:14.835849 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.835819    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 13:58:14.835849 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.835827    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 13:58:14.836615 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.836604    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 13:58:14.837843 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.837832    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 13:58:14.842500 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842476    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 13:58:14.842853 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842838    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 13:58:14.842917 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842886    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 13:58:14.842917 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842901    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 13:58:14.843010 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842924    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 13:58:14.843010 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842938    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 09 13:58:14.843010 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.842951    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 13:58:14.843339 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.843325    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 13:58:14.843413 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.843349    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 13:58:14.845705 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.845688    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 13:58:14.847937 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.847923    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 13:58:14.847994 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.847954    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 13:58:14.848796 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848784    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 13:58:14.848839 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848801    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 13:58:14.848839 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848807    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 13:58:14.848839 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848812    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 13:58:14.848839 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848826    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 13:58:14.848839 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848835    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848846    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848851    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848857    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848862    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848878    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 13:58:14.848971 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.848885    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 13:58:14.850748 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.850736    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 13:58:14.850808 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.850750    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 13:58:14.853562 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.853541    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-235.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 13:58:14.853867 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.853846    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 13:58:14.853908 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.853852    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 13:58:14.854199 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.854187    2580 server.go:1268] "Started kubelet"
Jul 09 13:58:14.854330 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.854296    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 13:58:14.854532 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.854278    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 13:58:14.854638 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.854594    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 13:58:14.855132 ip-10-0-143-235 systemd[1]: Started Kubernetes Kubelet.
Jul 09 13:58:14.859123 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.859100    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 13:58:14.860299 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.860277    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 13:58:14.860452 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.860438    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 13:58:14.867728 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.866604    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-235.ec2.internal.18c0a3bf643789b1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-235.ec2.internal,UID:ip-10-0-143-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-235.ec2.internal,},FirstTimestamp:2026-07-09 13:58:14.854158769 +0000 UTC m=+0.395425543,LastTimestamp:2026-07-09 13:58:14.854158769 +0000 UTC m=+0.395425543,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-235.ec2.internal,}"
Jul 09 13:58:14.868561 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.868540    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 13:58:14.868561 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.868551    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 13:58:14.868724 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.868629    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:14.869314 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869298    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 13:58:14.869409 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869329    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 13:58:14.869409 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869340    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 13:58:14.869623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869480    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 13:58:14.869623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869488    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 13:58:14.869623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869518    2580 factory.go:55] Registering systemd factory
Jul 09 13:58:14.869623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869535    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 09 13:58:14.869878 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869859    2580 factory.go:153] Registering CRI-O factory
Jul 09 13:58:14.869934 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869882    2580 factory.go:223] Registration of the crio container factory successfully
Jul 09 13:58:14.869976 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869933    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 13:58:14.869976 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869962    2580 factory.go:103] Registering Raw factory
Jul 09 13:58:14.870063 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.869977    2580 manager.go:1196] Started watching for new ooms in manager
Jul 09 13:58:14.870433 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.870418    2580 manager.go:349] Starting recovery of all containers
Jul 09 13:58:14.873829 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.873801    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 13:58:14.873908 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.873884    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-143-235.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 13:58:14.881022 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.880910    2580 manager.go:354] Recovery completed
Jul 09 13:58:14.885356 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.885341    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:14.887290 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.887268    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nfgv"
Jul 09 13:58:14.887444 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.887428    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:14.887512 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.887464    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:14.887512 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.887481    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:14.888002 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.887986    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 13:58:14.888052 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888003    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 13:58:14.888052 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888019    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 13:58:14.888142 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888058    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:14.888790 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888773    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:14.888847 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888806    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:14.888847 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.888820    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:14.889347 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.889263    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-235.ec2.internal.18c0a3bf66337960  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-235.ec2.internal,UID:ip-10-0-143-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-143-235.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-143-235.ec2.internal,},FirstTimestamp:2026-07-09 13:58:14.88744688 +0000 UTC m=+0.428713658,LastTimestamp:2026-07-09 13:58:14.88744688 +0000 UTC m=+0.428713658,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-235.ec2.internal,}"
Jul 09 13:58:14.890103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.890091    2580 policy_none.go:49] "None policy: Start"
Jul 09 13:58:14.890156 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.890108    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 13:58:14.890156 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.890118    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 13:58:14.891973 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.891962    2580 policy_none.go:47] "Start"
Jul 09 13:58:14.895182 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.895159    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8nfgv"
Jul 09 13:58:14.900631 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.900535    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-235.ec2.internal.18c0a3bf6633da96  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-235.ec2.internal,UID:ip-10-0-143-235.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-143-235.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-143-235.ec2.internal,},FirstTimestamp:2026-07-09 13:58:14.887471766 +0000 UTC m=+0.428738542,LastTimestamp:2026-07-09 13:58:14.887471766 +0000 UTC m=+0.428738542,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-235.ec2.internal,}"
Jul 09 13:58:14.935269 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935248    2580 manager.go:337] "Starting Device Plugin manager"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.935301    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935316    2580 server.go:87] "Starting device plugin registration server"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935542    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935552    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935616    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935704    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:14.935716    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.936195    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 13:58:14.951579 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:14.936243    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.036429 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.036347    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.040419 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.040381    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.040523 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.040425    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.040523 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.040436    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.040523 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.040463    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.052597 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.052554    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 13:58:15.053168 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.053154    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.053254 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.053179    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-235.ec2.internal\": node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.054031 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.054016    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 13:58:15.054103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.054057    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 13:58:15.054103 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.054079    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 13:58:15.054193 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.054117    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 13:58:15.056473 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.056456    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:58:15.075665 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.075642    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.154669 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.154633    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"]
Jul 09 13:58:15.154827 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.154715    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.155771 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.155745    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.155838 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.155780    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.155838 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.155794    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.158181 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.158167    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.158309 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.158293    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.158352 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.158339    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.159450 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159430    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.159503 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159447    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.159503 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159457    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.159503 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159467    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.159503 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159468    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.159503 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.159477    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.161706 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.161693    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.161762 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.161716    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.162554 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.162537    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.162652 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.162580    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.162652 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.162593    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.170941 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.170923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.171019 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.170948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.171019 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.170965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d5108fce62f539a837eb3b8262da6a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-235.ec2.internal\" (UID: \"2d5108fce62f539a837eb3b8262da6a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.176041 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.176024    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.177086 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.177072    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-235.ec2.internal\" not found" node="ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.180958 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.180945    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-235.ec2.internal\" not found" node="ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271238 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271238 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271238 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d5108fce62f539a837eb3b8262da6a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-235.ec2.internal\" (UID: \"2d5108fce62f539a837eb3b8262da6a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271486 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2d5108fce62f539a837eb3b8262da6a3-config\") pod \"kube-apiserver-proxy-ip-10-0-143-235.ec2.internal\" (UID: \"2d5108fce62f539a837eb3b8262da6a3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271486 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.271486 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.271327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e2eb795d63fb43ec9d4d6215dc882336-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal\" (UID: \"e2eb795d63fb43ec9d4d6215dc882336\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.276299 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.276280    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.377153 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.377092    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.477699 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.477662    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.478770 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.478755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.483917 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.483768    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:15.578145 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.578099    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.678731 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.678663    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.779328 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.779287    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.800470 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.800437    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 13:58:15.800652 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.800616    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 13:58:15.879541 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.879499    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.901717 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.901656    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 13:53:15 +0000 UTC" deadline="2028-03-07 20:54:00.434008986 +0000 UTC"
Jul 09 13:58:15.901717 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.901711    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14574h55m44.532301726s"
Jul 09 13:58:15.968015 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.967989    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 13:58:15.969105 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.969088    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 13:58:15.969207 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.969118    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 13:58:15.969207 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.969129    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeHasSufficientPID"
Jul 09 13:58:15.979643 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:15.979616    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-235.ec2.internal\" not found"
Jul 09 13:58:15.998542 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:15.998363    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:58:16.069473 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.069450    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:16.079495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.079473    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 13:58:16.080858 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:16.080826    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2eb795d63fb43ec9d4d6215dc882336.slice/crio-4b096866c645f197e734978ea9bc4d242418db6ca06f98f7c216d04792632cce WatchSource:0}: Error finding container 4b096866c645f197e734978ea9bc4d242418db6ca06f98f7c216d04792632cce: Status 404 returned error can't find the container with id 4b096866c645f197e734978ea9bc4d242418db6ca06f98f7c216d04792632cce
Jul 09 13:58:16.081097 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.081075    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal"
Jul 09 13:58:16.081201 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:16.081104    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d5108fce62f539a837eb3b8262da6a3.slice/crio-28b6c8e80c9de3f144ae920e9bb0fb94c36a4f8a4816990d3e636e4cc03cdbb8 WatchSource:0}: Error finding container 28b6c8e80c9de3f144ae920e9bb0fb94c36a4f8a4816990d3e636e4cc03cdbb8: Status 404 returned error can't find the container with id 28b6c8e80c9de3f144ae920e9bb0fb94c36a4f8a4816990d3e636e4cc03cdbb8
Jul 09 13:58:16.086391 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.086375    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 13:58:16.092585 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.092554    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 13:58:16.177979 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.177940    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:58:16.350414 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.350317    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:58:16.843949 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.843829    2580 apiserver.go:52] "Watching apiserver"
Jul 09 13:58:16.856877 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.856835    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 13:58:16.859043 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.859015    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-lbj27","openshift-network-diagnostics/network-check-target-pdv74","openshift-ovn-kubernetes/ovnkube-node-77rxp","kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km","openshift-dns/node-resolver-dxqhw","openshift-image-registry/node-ca-wkbvh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal","openshift-multus/multus-additional-cni-plugins-zjd4c","openshift-multus/network-metrics-daemon-nzsrt","openshift-network-operator/iptables-alerter-spts4","kube-system/global-pull-secret-syncer-x5q8v","kube-system/konnectivity-agent-hb59q","openshift-cluster-node-tuning-operator/tuned-7mxlm"]
Jul 09 13:58:16.862546 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.862204    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.862546 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:16.862302    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:16.864696 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.864671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.868043 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.868020    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6fc9n\""
Jul 09 13:58:16.868330 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.868312    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 13:58:16.868684 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.868666    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 13:58:16.868901 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.868888    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 13:58:16.872242 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.872222    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.875863 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.874809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.875863 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.875231    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.875863 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.875455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 13:58:16.875863 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.875511    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pcmzz\""
Jul 09 13:58:16.875863 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.875591    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 13:58:16.876376 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.876358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 13:58:16.876464 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.876441    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.877093 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.877077    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.877822 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.877799    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 13:58:16.877954 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.877935    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.878156 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.878137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.878741    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-flkmt\""
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.878980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-log-socket\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0d9197b-afa8-448e-a332-0db0c62e053c-ovn-node-metrics-cert\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-script-lib\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879095    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a03819f3-15f7-4c93-8b74-04c2cb175f88-konnectivity-ca\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-kubelet\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879147    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-netns\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-etc-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879198    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-node-log\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-netd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879281    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a03819f3-15f7-4c93-8b74-04c2cb175f88-agent-certs\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-systemd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-bin\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.879930 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-config\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-env-overrides\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shgb\" (UniqueName: \"kubernetes.io/projected/c0d9197b-afa8-448e-a332-0db0c62e053c-kube-api-access-2shgb\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879531    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-systemd-units\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-slash\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879598    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-var-lib-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-ovn\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvksf\" (UniqueName: \"kubernetes.io/projected/60bf2bd8-1417-48bb-93cd-c7e4763d6817-kube-api-access-jvksf\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.880698 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.879930    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:16.881265 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.880765    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.881265 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.881020    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 13:58:16.881265 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.881055    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 13:58:16.881439 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.881280    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-27n56\""
Jul 09 13:58:16.881504 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.881464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.882186 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.882167    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.883665 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.883631    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.883881 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.883862    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.885133 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-zpwdq\""
Jul 09 13:58:16.885231 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885132    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 13:58:16.885334 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885316    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 13:58:16.885393 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885368    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:16.885453 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885442    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.885581 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885565    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.885656 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885625    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k7tnw\""
Jul 09 13:58:16.885913 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.885896    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 13:58:16.887567 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.887535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:16.888032 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:16.887644    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:16.888310 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.888286    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.888565 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.888548    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 13:58:16.888966 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.888947    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.889178 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.889137    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nw8wd\""
Jul 09 13:58:16.891384 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.890492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.894765 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.893836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:16.894765 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.894353    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xxbvb\""
Jul 09 13:58:16.894765 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.894535    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 13:58:16.896897 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.896369    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 13:58:16.896897 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.896514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:16.896897 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:16.896582    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:16.896897 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.896684    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 13:58:16.896897 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.896725    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4jkf6\""
Jul 09 13:58:16.897271 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.896914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 13:58:16.907387 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.907356    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-h82zq"
Jul 09 13:58:16.917174 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.916997    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-h82zq"
Jul 09 13:58:16.970790 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.970766    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 13:58:16.980203 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cnibin\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.980321 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt2v6\" (UniqueName: \"kubernetes.io/projected/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-kube-api-access-kt2v6\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.980321 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980247    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-socket-dir-parent\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980321 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-hostroot\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980321 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-etc-kubernetes\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980321 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-tmp\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57169df3-14c5-4df9-990d-77581fa9941f-host\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980365    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-system-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-k8s-cni-cncf-io\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-kubelet\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-conf-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980503    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-kubelet\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-netd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980553    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57169df3-14c5-4df9-990d-77581fa9941f-serviceca\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980602    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-bin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.980617 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-kubelet\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a03819f3-15f7-4c93-8b74-04c2cb175f88-agent-certs\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-netd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-conf\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c5f22\" (UniqueName: \"kubernetes.io/projected/57169df3-14c5-4df9-990d-77581fa9941f-kube-api-access-c5f22\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-tuned\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9e0cf612-b40a-45f1-b115-3217ccda5d3f-iptables-alerter-script\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr4f2\" (UniqueName: \"kubernetes.io/projected/9e0cf612-b40a-45f1-b115-3217ccda5d3f-kube-api-access-dr4f2\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.980964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9swh\" (UniqueName: \"kubernetes.io/projected/61a41da2-22cb-4d21-9a91-87a2890e012e-kube-api-access-r9swh\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-log-socket\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981028    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0d9197b-afa8-448e-a332-0db0c62e053c-ovn-node-metrics-cert\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981050    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysconfig\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42df1555-d3fb-4246-807a-e18a880a86a5-hosts-file\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981129    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-log-socket\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-registration-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981208    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nz8k\" (UniqueName: \"kubernetes.io/projected/aaf4d1cb-9869-4307-93b4-7930812d4257-kube-api-access-7nz8k\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a03819f3-15f7-4c93-8b74-04c2cb175f88-konnectivity-ca\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981299    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-etc-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-dbus\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-etc-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-cni-binary-copy\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-netns\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-bin\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981461    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-sys-fs\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.981575 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981473    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-cni-bin\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981489    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-cnibin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-ovn\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-modprobe-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981588    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-855dq\" (UniqueName: \"kubernetes.io/projected/4388c2d6-e984-4421-ab4f-01f614d632b7-kube-api-access-855dq\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981621    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxpt7\" (UniqueName: \"kubernetes.io/projected/42df1555-d3fb-4246-807a-e18a880a86a5-kube-api-access-zxpt7\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-ovn\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-etc-selinux\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981719    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-script-lib\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-socket-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-multus\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42df1555-d3fb-4246-807a-e18a880a86a5-tmp-dir\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a03819f3-15f7-4c93-8b74-04c2cb175f88-konnectivity-ca\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:16.981902    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-kubelet-config\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:16.982634 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.981945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:16.981972    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:17.481948398 +0000 UTC m=+3.023215161 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvksf\" (UniqueName: \"kubernetes.io/projected/60bf2bd8-1417-48bb-93cd-c7e4763d6817-kube-api-access-jvksf\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-kubernetes\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-os-release\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-systemd-units\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-systemd\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-lib-modules\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-script-lib\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-var-lib-kubelet\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982436    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-systemd-units\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982505    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-daemon-config\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.983449 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-multus-certs\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982566    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-netns\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982590    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-node-log\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982650    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-run-netns\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-config\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-env-overrides\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-node-log\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982713    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-sys\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e0cf612-b40a-45f1-b115-3217ccda5d3f-host-slash\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-device-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-systemd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shgb\" (UniqueName: \"kubernetes.io/projected/c0d9197b-afa8-448e-a332-0db0c62e053c-kube-api-access-2shgb\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-run\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.984301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-host\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-system-cni-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.982998    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-os-release\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983025    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-slash\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-var-lib-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-env-overrides\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-var-lib-openvswitch\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983182    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c0d9197b-afa8-448e-a332-0db0c62e053c-ovnkube-config\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-run-systemd\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.983239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c0d9197b-afa8-448e-a332-0db0c62e053c-host-slash\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985119 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.984881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c0d9197b-afa8-448e-a332-0db0c62e053c-ovn-node-metrics-cert\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:16.985704 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.985681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a03819f3-15f7-4c93-8b74-04c2cb175f88-agent-certs\") pod \"konnectivity-agent-hb59q\" (UID: \"a03819f3-15f7-4c93-8b74-04c2cb175f88\") " pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:16.994657 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.994634    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvksf\" (UniqueName: \"kubernetes.io/projected/60bf2bd8-1417-48bb-93cd-c7e4763d6817-kube-api-access-jvksf\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:16.995885 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:16.995864    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shgb\" (UniqueName: \"kubernetes.io/projected/c0d9197b-afa8-448e-a332-0db0c62e053c-kube-api-access-2shgb\") pod \"ovnkube-node-77rxp\" (UID: \"c0d9197b-afa8-448e-a332-0db0c62e053c\") " pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:17.060794 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.060710    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal" event={"ID":"2d5108fce62f539a837eb3b8262da6a3","Type":"ContainerStarted","Data":"28b6c8e80c9de3f144ae920e9bb0fb94c36a4f8a4816990d3e636e4cc03cdbb8"}
Jul 09 13:58:17.062082 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.062048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal" event={"ID":"e2eb795d63fb43ec9d4d6215dc882336","Type":"ContainerStarted","Data":"4b096866c645f197e734978ea9bc4d242418db6ca06f98f7c216d04792632cce"}
Jul 09 13:58:17.083654 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083606    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.083654 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-daemon-config\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.083654 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-multus-certs\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-sys\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e0cf612-b40a-45f1-b115-3217ccda5d3f-host-slash\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-device-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-multus-certs\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083805    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-sys\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-run\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083850    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-host\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-device-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-system-cni-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.083911 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e0cf612-b40a-45f1-b115-3217ccda5d3f-host-slash\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-host\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-system-cni-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084006    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-os-release\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.083936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-os-release\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084029    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-run\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.084033    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cnibin\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt2v6\" (UniqueName: \"kubernetes.io/projected/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-kube-api-access-kt2v6\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.084156    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:17.58409438 +0000 UTC m=+3.125361147 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-socket-dir-parent\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-hostroot\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-etc-kubernetes\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-tmp\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57169df3-14c5-4df9-990d-77581fa9941f-host\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084293    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-hostroot\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084310    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-socket-dir-parent\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.084428 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cnibin\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084362    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-etc-kubernetes\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-system-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-system-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-daemon-config\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/57169df3-14c5-4df9-990d-77581fa9941f-host\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084406    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-k8s-cni-cncf-io\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-k8s-cni-cncf-io\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-kubelet\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-conf-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57169df3-14c5-4df9-990d-77581fa9941f-serviceca\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084601    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-bin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-conf-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-conf\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c5f22\" (UniqueName: \"kubernetes.io/projected/57169df3-14c5-4df9-990d-77581fa9941f-kube-api-access-c5f22\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-tuned\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.085540 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9e0cf612-b40a-45f1-b115-3217ccda5d3f-iptables-alerter-script\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr4f2\" (UniqueName: \"kubernetes.io/projected/9e0cf612-b40a-45f1-b115-3217ccda5d3f-kube-api-access-dr4f2\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-bin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9swh\" (UniqueName: \"kubernetes.io/projected/61a41da2-22cb-4d21-9a91-87a2890e012e-kube-api-access-r9swh\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysconfig\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084908    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysctl-conf\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-sysconfig\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/57169df3-14c5-4df9-990d-77581fa9941f-serviceca\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.084603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-kubelet\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42df1555-d3fb-4246-807a-e18a880a86a5-hosts-file\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-registration-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nz8k\" (UniqueName: \"kubernetes.io/projected/aaf4d1cb-9869-4307-93b4-7930812d4257-kube-api-access-7nz8k\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-dbus\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-registration-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085306    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/42df1555-d3fb-4246-807a-e18a880a86a5-hosts-file\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.086442 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-cni-binary-copy\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-netns\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-sys-fs\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-cnibin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085442    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-modprobe-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085447    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9e0cf612-b40a-45f1-b115-3217ccda5d3f-iptables-alerter-script\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-855dq\" (UniqueName: \"kubernetes.io/projected/4388c2d6-e984-4421-ab4f-01f614d632b7-kube-api-access-855dq\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085475    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxpt7\" (UniqueName: \"kubernetes.io/projected/42df1555-d3fb-4246-807a-e18a880a86a5-kube-api-access-zxpt7\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-etc-selinux\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085553    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-socket-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085572    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-dbus\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-multus\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085584    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-modprobe-d\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-sys-fs\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42df1555-d3fb-4246-807a-e18a880a86a5-tmp-dir\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085704    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-kubelet-config\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.087361 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-cnibin\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-kubernetes\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085812    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-kubernetes\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-os-release\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085910    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-systemd\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-lib-modules\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aaf4d1cb-9869-4307-93b4-7930812d4257-cni-binary-copy\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-var-lib-kubelet\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.085999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.086004    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-os-release\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.086063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-run-netns\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.086695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-cni-binary-copy\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.086813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-etc-selinux\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.086970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-lib-modules\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-systemd\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4388c2d6-e984-4421-ab4f-01f614d632b7-var-lib-kubelet\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.088160 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-host-var-lib-cni-multus\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/05625673-1f39-4a55-9a8a-0c55f5245758-kubelet-config\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aaf4d1cb-9869-4307-93b4-7930812d4257-multus-cni-dir\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/61a41da2-22cb-4d21-9a91-87a2890e012e-socket-dir\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.089024 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.087623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/42df1555-d3fb-4246-807a-e18a880a86a5-tmp-dir\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.090131 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.090105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-tmp\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.090262 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.090236    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4388c2d6-e984-4421-ab4f-01f614d632b7-etc-tuned\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.106667 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.106559    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:17.106667 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.106610    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:17.106667 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.106628    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:17.106950 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.106716    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:17.60669407 +0000 UTC m=+3.147960850 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:17.107568 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.107540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c5f22\" (UniqueName: \"kubernetes.io/projected/57169df3-14c5-4df9-990d-77581fa9941f-kube-api-access-c5f22\") pod \"node-ca-wkbvh\" (UID: \"57169df3-14c5-4df9-990d-77581fa9941f\") " pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.108803 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.108479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr4f2\" (UniqueName: \"kubernetes.io/projected/9e0cf612-b40a-45f1-b115-3217ccda5d3f-kube-api-access-dr4f2\") pod \"iptables-alerter-spts4\" (UID: \"9e0cf612-b40a-45f1-b115-3217ccda5d3f\") " pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.108803 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.108633    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nz8k\" (UniqueName: \"kubernetes.io/projected/aaf4d1cb-9869-4307-93b4-7930812d4257-kube-api-access-7nz8k\") pod \"multus-lbj27\" (UID: \"aaf4d1cb-9869-4307-93b4-7930812d4257\") " pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.108803 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.108755    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9swh\" (UniqueName: \"kubernetes.io/projected/61a41da2-22cb-4d21-9a91-87a2890e012e-kube-api-access-r9swh\") pod \"aws-ebs-csi-driver-node-mx2km\" (UID: \"61a41da2-22cb-4d21-9a91-87a2890e012e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.109029 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.108851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxpt7\" (UniqueName: \"kubernetes.io/projected/42df1555-d3fb-4246-807a-e18a880a86a5-kube-api-access-zxpt7\") pod \"node-resolver-dxqhw\" (UID: \"42df1555-d3fb-4246-807a-e18a880a86a5\") " pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.109383 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.109354    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-855dq\" (UniqueName: \"kubernetes.io/projected/4388c2d6-e984-4421-ab4f-01f614d632b7-kube-api-access-855dq\") pod \"tuned-7mxlm\" (UID: \"4388c2d6-e984-4421-ab4f-01f614d632b7\") " pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.110235 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.110215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt2v6\" (UniqueName: \"kubernetes.io/projected/1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8-kube-api-access-kt2v6\") pod \"multus-additional-cni-plugins-zjd4c\" (UID: \"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8\") " pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.185673 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.185624    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:17.194792 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.194767    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:17.204851 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.204825    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7mxlm"
Jul 09 13:58:17.216866 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.216831    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km"
Jul 09 13:58:17.232075 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.232044    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-dxqhw"
Jul 09 13:58:17.243324 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.243297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-zjd4c"
Jul 09 13:58:17.251526 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.251500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wkbvh"
Jul 09 13:58:17.261437 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.261417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lbj27"
Jul 09 13:58:17.269430 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.269405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-spts4"
Jul 09 13:58:17.327739 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.327698    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 13:58:17.489688 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.489572    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:17.489858 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.489767    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:17.489858 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.489852    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:18.489827673 +0000 UTC m=+4.031094439 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:17.590802 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.590754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:17.591019 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.590943    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:17.591089 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.591038    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:18.591011836 +0000 UTC m=+4.132278614 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:17.691528 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.691488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:17.691730 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.691710    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:17.691778 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.691738    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:17.691778 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.691748    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:17.691841 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:17.691810    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:18.691791097 +0000 UTC m=+4.233057862 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:17.922261 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.922224    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:53:17 +0000 UTC" deadline="2028-01-02 07:19:33.783368484 +0000 UTC"
Jul 09 13:58:17.922261 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:17.922257    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13001h21m15.861115771s"
Jul 09 13:58:18.129423 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:18.129228    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d9197b_afa8_448e_a332_0db0c62e053c.slice/crio-c5984a3f0c1f175bdf6e56b9f72394049cad47dd801c721957c5c7ec6f2678f1 WatchSource:0}: Error finding container c5984a3f0c1f175bdf6e56b9f72394049cad47dd801c721957c5c7ec6f2678f1: Status 404 returned error can't find the container with id c5984a3f0c1f175bdf6e56b9f72394049cad47dd801c721957c5c7ec6f2678f1
Jul 09 13:58:18.131399 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:18.131370    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a14ee55_2bdd_4d4c_ad81_268c6a9d67a8.slice/crio-070d909179bfc6c0975ae690f41b2837ca8b0cb4272218722bebd5fbf50bdc0f WatchSource:0}: Error finding container 070d909179bfc6c0975ae690f41b2837ca8b0cb4272218722bebd5fbf50bdc0f: Status 404 returned error can't find the container with id 070d909179bfc6c0975ae690f41b2837ca8b0cb4272218722bebd5fbf50bdc0f
Jul 09 13:58:18.156422 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:18.156377    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4388c2d6_e984_4421_ab4f_01f614d632b7.slice/crio-372527d6a7f40cf9a3c4e2f5680da1ed5e19e2ec72a020e0213f53e517f7a1b7 WatchSource:0}: Error finding container 372527d6a7f40cf9a3c4e2f5680da1ed5e19e2ec72a020e0213f53e517f7a1b7: Status 404 returned error can't find the container with id 372527d6a7f40cf9a3c4e2f5680da1ed5e19e2ec72a020e0213f53e517f7a1b7
Jul 09 13:58:18.157142 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:18.157114    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42df1555_d3fb_4246_807a_e18a880a86a5.slice/crio-255fcb411105be3eed06e1d09bdf2d531fbbc033052b31c5966e51b49d4d2f2f WatchSource:0}: Error finding container 255fcb411105be3eed06e1d09bdf2d531fbbc033052b31c5966e51b49d4d2f2f: Status 404 returned error can't find the container with id 255fcb411105be3eed06e1d09bdf2d531fbbc033052b31c5966e51b49d4d2f2f
Jul 09 13:58:18.500436 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:18.500392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:18.500641 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.500518    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:18.500641 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.500575    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:20.500561402 +0000 UTC m=+6.041828166 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:18.600857 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:18.600811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:18.601043 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.600997    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:18.601122 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.601080    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:20.601056854 +0000 UTC m=+6.142323635 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:18.701977 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:18.701888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:18.702146 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.702066    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:18.702146 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.702087    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:18.702146 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.702101    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:18.702301 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:18.702166    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:20.702146468 +0000 UTC m=+6.243413258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:18.922658 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:18.922540    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 13:53:17 +0000 UTC" deadline="2028-04-19 21:03:34.170613862 +0000 UTC"
Jul 09 13:58:18.922658 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:18.922625    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15607h5m15.247993966s"
Jul 09 13:58:19.055867 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.055778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:19.056050 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:19.055921    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:19.056385 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.056362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:19.056483 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:19.056468    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:19.058124 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.058102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:19.058249 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:19.058224    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:19.078023 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.077984    2580 generic.go:356] "Generic (PLEG): container finished" podID="e2eb795d63fb43ec9d4d6215dc882336" containerID="779b63b520a106402ad14a5df9d2d7380150e06db5c3af6f815488d3ac86fe53" exitCode=0
Jul 09 13:58:19.078204 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.078093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal" event={"ID":"e2eb795d63fb43ec9d4d6215dc882336","Type":"ContainerDied","Data":"779b63b520a106402ad14a5df9d2d7380150e06db5c3af6f815488d3ac86fe53"}
Jul 09 13:58:19.085666 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.085625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerStarted","Data":"070d909179bfc6c0975ae690f41b2837ca8b0cb4272218722bebd5fbf50bdc0f"}
Jul 09 13:58:19.103843 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.103744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hb59q" event={"ID":"a03819f3-15f7-4c93-8b74-04c2cb175f88","Type":"ContainerStarted","Data":"5a8535bbaf4f8082b018e7a3abd42e7a8f662f0a8848cee66c99b431afb068fb"}
Jul 09 13:58:19.107700 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.107583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbj27" event={"ID":"aaf4d1cb-9869-4307-93b4-7930812d4257","Type":"ContainerStarted","Data":"6f9b773cf47d442b08db3f73af504fab4e894b3ec8805d6fe01606fa132672a5"}
Jul 09 13:58:19.120902 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.120108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal" event={"ID":"2d5108fce62f539a837eb3b8262da6a3","Type":"ContainerStarted","Data":"a0277d23941811f2ac17060f37f92da0f8d3e4cdc13f50604f053e691b7b5b5d"}
Jul 09 13:58:19.131491 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.131449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7mxlm" event={"ID":"4388c2d6-e984-4421-ab4f-01f614d632b7","Type":"ContainerStarted","Data":"372527d6a7f40cf9a3c4e2f5680da1ed5e19e2ec72a020e0213f53e517f7a1b7"}
Jul 09 13:58:19.147229 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.147166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dxqhw" event={"ID":"42df1555-d3fb-4246-807a-e18a880a86a5","Type":"ContainerStarted","Data":"255fcb411105be3eed06e1d09bdf2d531fbbc033052b31c5966e51b49d4d2f2f"}
Jul 09 13:58:19.161785 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.161743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"c5984a3f0c1f175bdf6e56b9f72394049cad47dd801c721957c5c7ec6f2678f1"}
Jul 09 13:58:19.187480 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.187437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-spts4" event={"ID":"9e0cf612-b40a-45f1-b115-3217ccda5d3f","Type":"ContainerStarted","Data":"f7c8a5331f6b4adcb30c209da41529941c5a9141a35a842d3711ac9ea932e592"}
Jul 09 13:58:19.213229 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.213182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km" event={"ID":"61a41da2-22cb-4d21-9a91-87a2890e012e","Type":"ContainerStarted","Data":"de89c037f90beacdff5d0df9a28deb1c7c175e1c2c3418014e305a84f25ac14b"}
Jul 09 13:58:19.230524 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:19.230434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkbvh" event={"ID":"57169df3-14c5-4df9-990d-77581fa9941f","Type":"ContainerStarted","Data":"89ca50b1367106033dd8538942ff7dd500f3c561ce7226d15a5978a471809239"}
Jul 09 13:58:20.251204 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:20.251111    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal" event={"ID":"e2eb795d63fb43ec9d4d6215dc882336","Type":"ContainerStarted","Data":"e2c77ef9a4804a014ec72480abf7361e12798e829fce7749bd97f014eeffb9c0"}
Jul 09 13:58:20.271514 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:20.271199    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-235.ec2.internal" podStartSLOduration=4.271173063 podStartE2EDuration="4.271173063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:19.137952111 +0000 UTC m=+4.679218917" watchObservedRunningTime="2026-07-09 13:58:20.271173063 +0000 UTC m=+5.812439850"
Jul 09 13:58:20.517876 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:20.517768    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:20.518063 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.517939    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:20.518063 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.518011    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:24.517991071 +0000 UTC m=+10.059257853 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:20.618280 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:20.618233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:20.618491 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.618432    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:20.618562 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.618532    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:24.618511215 +0000 UTC m=+10.159777993 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:20.719065 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:20.719020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:20.719249 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.719216    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:20.719249 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.719237    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:20.719249 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.719251    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:20.719433 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:20.719314    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:24.71929409 +0000 UTC m=+10.260560855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:21.055771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:21.055944    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:21.056318    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:21.056427    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:21.056476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:21.056611 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:21.056550    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:23.056190 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:23.056158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:23.056690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:23.056158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:23.056690 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:23.056301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:23.056690 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:23.056364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:23.056690 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:23.056467    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:23.056690 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:23.056553    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:24.553986 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:24.553943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:24.554416 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.554167    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:24.554416 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.554245    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:32.554223446 +0000 UTC m=+18.095490221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:24.655468 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:24.655425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:24.655683 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.655609    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:24.655753 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.655685    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:32.655663684 +0000 UTC m=+18.196930451 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:24.756762 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:24.756154    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:24.756762 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.756333    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:24.756762 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.756354    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:24.756762 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.756365    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:24.756762 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:24.756419    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:32.756402391 +0000 UTC m=+18.297669152 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:25.055819 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:25.055782    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:25.055976 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:25.055886    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:25.056213 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:25.056187    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:25.056349 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:25.056273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:25.056349 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:25.056319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:25.056457 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:25.056375    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:27.054683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:27.054822    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:27.055224    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:27.055310    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:27.055374    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:27.056064 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:27.055457    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:29.055263 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:29.055228    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:29.055711 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:29.055236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:29.055711 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:29.055358    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:29.055711 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:29.055236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:29.055711 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:29.055478    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:29.055711 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:29.055593    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:31.054532 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:31.054476    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:31.054993 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:31.054547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:31.054993 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:31.054560    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:31.054993 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:31.054677    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:31.054993 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:31.054718    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:31.054993 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:31.054839    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:32.617128 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:32.617086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:32.617694 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.617245    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:32.617694 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.617313    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:48.617292525 +0000 UTC m=+34.158559286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:32.718533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:32.718479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:32.718724 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.718626    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:32.718724 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.718687    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:48.718666909 +0000 UTC m=+34.259933671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:32.819785 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:32.819737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:32.819984 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.819952    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:32.819984 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.819978    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:32.820095 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.819991    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:32.820095 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:32.820063    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:58:48.820042552 +0000 UTC m=+34.361309318 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:33.057209 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:33.057127    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:33.057376 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:33.057230    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:33.057376 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:33.057244    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:33.057376 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:33.057256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:33.057376 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:33.057313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:33.057527 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:33.057400    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:35.056141 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:35.056096    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:35.056538 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:35.056199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:35.056538 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:35.056212    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:35.056538 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:35.056321    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:35.056538 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:35.056376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:35.056538 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:35.056446    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:37.054751 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:37.054717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:37.054751 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:37.054743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:37.055351 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:37.054726    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:37.055351 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:37.054848    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:37.055351 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:37.054918    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:37.055351 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:37.055017    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:38.290388 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.290019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7mxlm" event={"ID":"4388c2d6-e984-4421-ab4f-01f614d632b7","Type":"ContainerStarted","Data":"ac3499cdd98386fe54808cfb0250a991c021dbf08c668a583ca159bf7db047d5"}
Jul 09 13:58:38.291515 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.291484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-dxqhw" event={"ID":"42df1555-d3fb-4246-807a-e18a880a86a5","Type":"ContainerStarted","Data":"89ac3ea4c2cbe492a3bbdb959e5879afd721b2eab3c31d20878de769228fc33f"}
Jul 09 13:58:38.294246 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 13:58:38.294562 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294543    2580 generic.go:356] "Generic (PLEG): container finished" podID="c0d9197b-afa8-448e-a332-0db0c62e053c" containerID="752d01920042f9e2e8fc757efcfe91a4642f6f3d5f8b95aeedfc64d362b09608" exitCode=1
Jul 09 13:58:38.294649 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"a381ac82476160b157d2c2cc025b0ba73d9adec1dbb7ccb9843cc30a4782a6a3"}
Jul 09 13:58:38.294649 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"9351f2e9076aea3786d2401609981cd2e0d21c1d8aee3d5f4f613be93acc7d6f"}
Jul 09 13:58:38.294730 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294675    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"9c8560f1bad02b86ea0f71d5f2d67d0222f5996e88ccdccf7977887582b5120f"}
Jul 09 13:58:38.294730 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"75c22978c9aecfe915b16185023abf43b61700aa10145cbebf0954b3d42e5005"}
Jul 09 13:58:38.294804 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerDied","Data":"752d01920042f9e2e8fc757efcfe91a4642f6f3d5f8b95aeedfc64d362b09608"}
Jul 09 13:58:38.294804 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.294783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"f1c1c0e63f320595f7f3974d204d32523fdcc46c6bab6d38bd3165e6ff1e4199"}
Jul 09 13:58:38.295989 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.295970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km" event={"ID":"61a41da2-22cb-4d21-9a91-87a2890e012e","Type":"ContainerStarted","Data":"35aca454449c1285956b7752c59fccb419361a81522836da38f5ce3a2dd701a9"}
Jul 09 13:58:38.297244 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.297223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wkbvh" event={"ID":"57169df3-14c5-4df9-990d-77581fa9941f","Type":"ContainerStarted","Data":"f2d63311e3d58b12a4b79af54db6f986469c98439c78a611d0080a9504e993d2"}
Jul 09 13:58:38.298548 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.298525    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="449cbef025a7d013fd693eddb4bd61f0a568ea5f5c3198919ea0adaa15f341ba" exitCode=0
Jul 09 13:58:38.298641 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.298610    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"449cbef025a7d013fd693eddb4bd61f0a568ea5f5c3198919ea0adaa15f341ba"}
Jul 09 13:58:38.299983 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.299940    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hb59q" event={"ID":"a03819f3-15f7-4c93-8b74-04c2cb175f88","Type":"ContainerStarted","Data":"e17f4b06a0ad293c6b8d25c04cc95e3d59735ccb08862970460e1f093f917586"}
Jul 09 13:58:38.301656 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.301630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lbj27" event={"ID":"aaf4d1cb-9869-4307-93b4-7930812d4257","Type":"ContainerStarted","Data":"ea380a60ba4334e825004a844207d52d1518d15bb803a3963937df6183f18d1b"}
Jul 09 13:58:38.306130 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.306089    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7mxlm" podStartSLOduration=4.245402682 podStartE2EDuration="23.30607731s" totalImagesPullingTime="19.060674628s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:38.30594425 +0000 UTC m=+23.847211052" watchObservedRunningTime="2026-07-09 13:58:38.30607731 +0000 UTC m=+23.847344118"
Jul 09 13:58:38.306541 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.306520    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-235.ec2.internal" podStartSLOduration=22.306513384 podStartE2EDuration="22.306513384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:20.270349851 +0000 UTC m=+5.811616665" watchObservedRunningTime="2026-07-09 13:58:38.306513384 +0000 UTC m=+23.847780166"
Jul 09 13:58:38.321562 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.321513    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hb59q" podStartSLOduration=4.319161856 podStartE2EDuration="23.321498188s" totalImagesPullingTime="19.002336332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:38.321276244 +0000 UTC m=+23.862543026" watchObservedRunningTime="2026-07-09 13:58:38.321498188 +0000 UTC m=+23.862764970"
Jul 09 13:58:38.336378 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.336324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-dxqhw" podStartSLOduration=4.309898011 podStartE2EDuration="23.336311996s" totalImagesPullingTime="19.026413985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:38.335970756 +0000 UTC m=+23.877237552" watchObservedRunningTime="2026-07-09 13:58:38.336311996 +0000 UTC m=+23.877578779"
Jul 09 13:58:38.385932 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.385884    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wkbvh" podStartSLOduration=4.319762886 podStartE2EDuration="23.38586626s" totalImagesPullingTime="19.066103374s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:38.385367192 +0000 UTC m=+23.926633990" watchObservedRunningTime="2026-07-09 13:58:38.38586626 +0000 UTC m=+23.927133044"
Jul 09 13:58:38.401608 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.401534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lbj27" podStartSLOduration=4.007380043 podStartE2EDuration="23.401523234s" totalImagesPullingTime="19.394143191s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:38.401392813 +0000 UTC m=+23.942659595" watchObservedRunningTime="2026-07-09 13:58:38.401523234 +0000 UTC m=+23.942790017"
Jul 09 13:58:38.601218 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.601187    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 13:58:38.950790 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.950640    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T13:58:38.601204853Z","UUID":"110e113e-1e32-4f27-91b5-c29bd1ae316c","Handler":null,"Name":"","Endpoint":""}
Jul 09 13:58:38.952188 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.952164    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 13:58:38.952278 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:38.952206    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.055373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.055448    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.055449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:39.055558    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:39.055670    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:39.055838 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:39.055753    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:39.306217 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.306122    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-spts4" event={"ID":"9e0cf612-b40a-45f1-b115-3217ccda5d3f","Type":"ContainerStarted","Data":"1f003bd50f5a411bc76409e9fa963ae57dce3c5c1a5a86ff9eb88992c11f2956"}
Jul 09 13:58:39.308236 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.308208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km" event={"ID":"61a41da2-22cb-4d21-9a91-87a2890e012e","Type":"ContainerStarted","Data":"996632ee2085e81833f857e83591215d09d36ddc1ea112a0637dc53c231699a4"}
Jul 09 13:58:39.321236 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:39.321187    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-spts4" podStartSLOduration=5.234628307 podStartE2EDuration="24.321171135s" totalImagesPullingTime="19.086542828s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:39.320684141 +0000 UTC m=+24.861950923" watchObservedRunningTime="2026-07-09 13:58:39.321171135 +0000 UTC m=+24.862437940"
Jul 09 13:58:40.085490 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.085283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:40.085984 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.085964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:40.316089 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.316060    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 13:58:40.316994 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.316950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"7347075e8f08ca8e33ee6648a9f5304f0f3f407fa618998bdebacc5389849cfa"}
Jul 09 13:58:40.319408 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.319377    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km" event={"ID":"61a41da2-22cb-4d21-9a91-87a2890e012e","Type":"ContainerStarted","Data":"9adb416057d2b1e78a11829b9bedcddeea2ffbeb0762df4b1dddf22ac2b66884"}
Jul 09 13:58:40.336533 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:40.336443    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mx2km" podStartSLOduration=4.5030750699999995 podStartE2EDuration="25.336422842s" totalImagesPullingTime="20.833347772s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:40.336141604 +0000 UTC m=+25.877408386" watchObservedRunningTime="2026-07-09 13:58:40.336422842 +0000 UTC m=+25.877689637"
Jul 09 13:58:41.054752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:41.054705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:41.054949 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:41.054712    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:41.054949 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:41.054830    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:41.054949 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:41.054718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:41.054949 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:41.054901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:41.055162 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:41.054991    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:41.325236 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:41.325144    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 13:58:42.758466 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:42.758418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:42.759056 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:42.758612    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 09 13:58:42.759639 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:42.759616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hb59q"
Jul 09 13:58:43.055153 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:43.055071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:43.055153 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:43.055106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:43.055153 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:43.055071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:43.055423 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:43.055197    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:43.055423 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:43.055286    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:43.055423 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:43.055373    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:44.334448 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.334278    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 13:58:44.335081 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.334825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"07a8677a50c58a921db2eb278c603bab27d919874eca68b4a3e9632c8c247f84"}
Jul 09 13:58:44.335081 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.335070    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:44.335205 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.335138    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:44.335258 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.335242    2580 scope.go:117] "RemoveContainer" containerID="752d01920042f9e2e8fc757efcfe91a4642f6f3d5f8b95aeedfc64d362b09608"
Jul 09 13:58:44.336809 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.336784    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="ab03692b9945f6643ac283f1f02917297b3c08b5934a918afb2ea36ed90d777f" exitCode=0
Jul 09 13:58:44.336903 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.336823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"ab03692b9945f6643ac283f1f02917297b3c08b5934a918afb2ea36ed90d777f"}
Jul 09 13:58:44.353645 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:44.353621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:45.058338 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.058309    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:45.058506 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.058422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:45.058506 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.058432    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:45.058506 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.058460    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:45.058700 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.058590    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:45.058700 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.058689    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:45.342972 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.342813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 13:58:45.343646 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.343403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" event={"ID":"c0d9197b-afa8-448e-a332-0db0c62e053c","Type":"ContainerStarted","Data":"b1acba00eb5b16badca5e700c87132852a43ee0523515771cdbe96c1827b204d"}
Jul 09 13:58:45.343646 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.343640    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:45.345862 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.345832    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="f5367d6fcef4e59c1acb6a86a67db7a116f0bd52f3ec21be44f5a9d940e960f6" exitCode=0
Jul 09 13:58:45.345951 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.345885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"f5367d6fcef4e59c1acb6a86a67db7a116f0bd52f3ec21be44f5a9d940e960f6"}
Jul 09 13:58:45.360937 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.360911    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:58:45.372176 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.372128    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp" podStartSLOduration=11.255145669000001 podStartE2EDuration="30.372112055s" totalImagesPullingTime="19.116966386s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:45.371529305 +0000 UTC m=+30.912796088" watchObservedRunningTime="2026-07-09 13:58:45.372112055 +0000 UTC m=+30.913378854"
Jul 09 13:58:45.749034 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.748950    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdv74"]
Jul 09 13:58:45.749187 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.749126    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:45.749271 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.749242    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:45.749497 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.749477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5q8v"]
Jul 09 13:58:45.749599 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.749587    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:45.749682 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.749666    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:45.751622 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.751600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nzsrt"]
Jul 09 13:58:45.751719 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:45.751710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:45.751838 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:45.751821    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:46.351062 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:46.351032    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="a1e766afa9d70f8d4b54b30f7d4ce32943dead5dc4aa3d16def4b90d1864e9e5" exitCode=0
Jul 09 13:58:46.351501 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:46.351097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"a1e766afa9d70f8d4b54b30f7d4ce32943dead5dc4aa3d16def4b90d1864e9e5"}
Jul 09 13:58:47.055304 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:47.055273    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:47.055456 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:47.055417    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:48.055042 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:48.055005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:48.055797 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:48.055056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:48.055797 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.055188    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:48.055797 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.055301    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:48.644560 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:48.644521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:48.644788 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.644699    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:48.644788 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.644761    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:20.644745707 +0000 UTC m=+66.186012468 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 13:58:48.745819 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:48.745782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:48.745965 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.745936    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:48.746011 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.746000    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret podName:05625673-1f39-4a55-9a8a-0c55f5245758 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:20.745984942 +0000 UTC m=+66.287251702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret") pod "global-pull-secret-syncer-x5q8v" (UID: "05625673-1f39-4a55-9a8a-0c55f5245758") : object "kube-system"/"original-pull-secret" not registered
Jul 09 13:58:48.847143 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:48.847104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:48.847315 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.847271    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 13:58:48.847315 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.847292    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 13:58:48.847315 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.847301    2580 projected.go:196] Error preparing data for projected volume kube-api-access-5ts48 for pod openshift-network-diagnostics/network-check-target-pdv74: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:48.847402 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:48.847357    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48 podName:74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:20.847341083 +0000 UTC m=+66.388607864 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5ts48" (UniqueName: "kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48") pod "network-check-target-pdv74" (UID: "74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 13:58:49.054605 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:49.054555    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:49.054769 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:49.054683    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x5q8v" podUID="05625673-1f39-4a55-9a8a-0c55f5245758"
Jul 09 13:58:50.054476 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.054439    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:50.054883 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.054485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:50.054883 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.054588    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nzsrt" podUID="60bf2bd8-1417-48bb-93cd-c7e4763d6817"
Jul 09 13:58:50.054883 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.054687    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pdv74" podUID="74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6"
Jul 09 13:58:50.212279 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.212209    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-235.ec2.internal" event="NodeReady"
Jul 09 13:58:50.212431 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.212359    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 13:58:50.274900 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.274863    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 13:58:50.297946 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.297363    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xlsp9"]
Jul 09 13:58:50.297946 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.297479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.300219 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.300170    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 13:58:50.300405 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.300303    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 13:58:50.300507 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.300424    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 13:58:50.301491 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.301465    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n8xtq\""
Jul 09 13:58:50.306999 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.306974    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 13:58:50.315298 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.315279    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-t6xq5"]
Jul 09 13:58:50.315619 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.315423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.318083 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.318057    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 13:58:50.318185 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.318109    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 13:58:50.318185 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.318133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hf6lw\""
Jul 09 13:58:50.328121 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.328099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 13:58:50.328248 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.328135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xlsp9"]
Jul 09 13:58:50.328248 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.328156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t6xq5"]
Jul 09 13:58:50.328248 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.328232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.330939 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.330921    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 13:58:50.331036 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.330956    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 13:58:50.331036 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.330996    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jssv4\""
Jul 09 13:58:50.331149 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.330930    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 13:58:50.460296 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460268    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460296 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjqcc\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/855db753-1b10-4049-a83b-e9b1b989775a-config-volume\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnsd2\" (UniqueName: \"kubernetes.io/projected/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-kube-api-access-bnsd2\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5jzn\" (UniqueName: \"kubernetes.io/projected/855db753-1b10-4049-a83b-e9b1b989775a-kube-api-access-b5jzn\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.460495 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460444    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-cert\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.460752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.460752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.460752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855db753-1b10-4049-a83b-e9b1b989775a-tmp-dir\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.460752 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.460648    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.561681 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.561681 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5jzn\" (UniqueName: \"kubernetes.io/projected/855db753-1b10-4049-a83b-e9b1b989775a-kube-api-access-b5jzn\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561721    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-cert\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855db753-1b10-4049-a83b-e9b1b989775a-tmp-dir\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.561905 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561875    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561913    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.561823    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.561959    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.562025    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:58:51.062005535 +0000 UTC m=+36.603272321 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.562030    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.561935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjqcc\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:50.562070    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:58:51.062056676 +0000 UTC m=+36.603323437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/855db753-1b10-4049-a83b-e9b1b989775a-config-volume\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562137    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562168    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562189 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnsd2\" (UniqueName: \"kubernetes.io/projected/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-kube-api-access-bnsd2\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.562739 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.562739 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/855db753-1b10-4049-a83b-e9b1b989775a-tmp-dir\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.562869 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/855db753-1b10-4049-a83b-e9b1b989775a-config-volume\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.563039 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.562928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.563365 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.563343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.566510 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.566488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.566510 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.566504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.566700 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.566680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-cert\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.573422 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.573294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnsd2\" (UniqueName: \"kubernetes.io/projected/1e20d41c-0a27-40b7-a77f-3b3d88e9979a-kube-api-access-bnsd2\") pod \"ingress-canary-t6xq5\" (UID: \"1e20d41c-0a27-40b7-a77f-3b3d88e9979a\") " pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:50.573707 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.573689    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjqcc\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.573815 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.573789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:50.574276 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.574258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5jzn\" (UniqueName: \"kubernetes.io/projected/855db753-1b10-4049-a83b-e9b1b989775a-kube-api-access-b5jzn\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:50.638651 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:50.638621    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-t6xq5"
Jul 09 13:58:51.054611 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:51.054562    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:58:51.057505 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:51.057483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 13:58:51.067630 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:51.067611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:51.067761 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:51.067670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:51.067820 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:51.067755    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:58:51.067820 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:51.067771    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:58:51.067923 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:51.067818    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:51.067923 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:51.067829    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:58:52.067811389 +0000 UTC m=+37.609078166 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:58:51.067923 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:51.067867    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:58:52.067852343 +0000 UTC m=+37.609119104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:58:52.054641 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.054610    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:58:52.055015 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.054651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:58:52.057497 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.057478    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 13:58:52.057608 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.057483    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 13:58:52.058249 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.058231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qwpwr\""
Jul 09 13:58:52.058367 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.058256    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 13:58:52.058367 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.058237    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2p5t2\""
Jul 09 13:58:52.074999 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.074978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:52.075125 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.075023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:52.075125 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:52.075111    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:52.075125 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:52.075113    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:58:52.075272 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:52.075134    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:58:52.075272 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:52.075158    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:58:54.075145979 +0000 UTC m=+39.616412740 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:58:52.075272 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:52.075181    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:58:54.075168505 +0000 UTC m=+39.616435266 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:58:52.195090 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.195030    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-t6xq5"]
Jul 09 13:58:52.256187 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:58:52.256153    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e20d41c_0a27_40b7_a77f_3b3d88e9979a.slice/crio-02299e2217ef54ae26b7f79b4c61c6875972075956ccdedaef23e42d1b9354a5 WatchSource:0}: Error finding container 02299e2217ef54ae26b7f79b4c61c6875972075956ccdedaef23e42d1b9354a5: Status 404 returned error can't find the container with id 02299e2217ef54ae26b7f79b4c61c6875972075956ccdedaef23e42d1b9354a5
Jul 09 13:58:52.369087 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:52.369055    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t6xq5" event={"ID":"1e20d41c-0a27-40b7-a77f-3b3d88e9979a","Type":"ContainerStarted","Data":"02299e2217ef54ae26b7f79b4c61c6875972075956ccdedaef23e42d1b9354a5"}
Jul 09 13:58:53.374587 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:53.374546    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="18c74d16939adc225b86c91b48cbda8318deb5c2d11b0a013fcee59d3fdc9caa" exitCode=0
Jul 09 13:58:53.375034 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:53.374602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"18c74d16939adc225b86c91b48cbda8318deb5c2d11b0a013fcee59d3fdc9caa"}
Jul 09 13:58:54.093350 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:54.093150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:54.093606 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:54.093412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:54.093606 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:54.093313    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:54.093606 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:54.093504    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:58:58.093484 +0000 UTC m=+43.634750763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:58:54.093606 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:54.093537    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:58:54.093606 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:54.093551    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:58:54.093820 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:54.093623    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:58:58.093609284 +0000 UTC m=+43.634876045 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:58:54.380714 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:54.380624    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8" containerID="bd0740ef7266cc487362da7db8763e347abc561f32e9dbcc83a0c8f518342a15" exitCode=0
Jul 09 13:58:54.380714 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:54.380674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerDied","Data":"bd0740ef7266cc487362da7db8763e347abc561f32e9dbcc83a0c8f518342a15"}
Jul 09 13:58:55.386952 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:55.386914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" event={"ID":"1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8","Type":"ContainerStarted","Data":"565afe2143ffb9fc47c921d5ee871b236c72f94389c0623be96825567ec08be6"}
Jul 09 13:58:55.388343 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:55.388316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-t6xq5" event={"ID":"1e20d41c-0a27-40b7-a77f-3b3d88e9979a","Type":"ContainerStarted","Data":"a973beb26902eb3eb69e0804281815251f876ea000fb0f66b8b27622e30af7fe"}
Jul 09 13:58:55.409371 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:55.409322    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-zjd4c" podStartSLOduration=8.529074574 podStartE2EDuration="40.409304755s" totalImagesPullingTime="31.880230181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:55.408388928 +0000 UTC m=+40.949655708" watchObservedRunningTime="2026-07-09 13:58:55.409304755 +0000 UTC m=+40.950571538"
Jul 09 13:58:55.423359 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:55.423309    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-t6xq5" podStartSLOduration=2.698272888 podStartE2EDuration="5.423293126s" totalImagesPullingTime="2.725020238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:58:55.421812223 +0000 UTC m=+40.963079031" watchObservedRunningTime="2026-07-09 13:58:55.423293126 +0000 UTC m=+40.964559908"
Jul 09 13:58:58.125434 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:58.125395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:58:58.125466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:58.125544    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:58.125593    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:58.125605    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:58.125646    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:59:06.125622535 +0000 UTC m=+51.666889299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:58:58.125952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:58:58.125665    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:59:06.125656485 +0000 UTC m=+51.666923245 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:59:06.186835 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:06.186767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:06.186882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:06.186926    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:06.186968    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:06.186978    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:06.186996    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:59:22.186979968 +0000 UTC m=+67.728246731 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:59:06.187294 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:06.187030    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:59:22.187015997 +0000 UTC m=+67.728282758 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:59:17.364185 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:17.364158    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-77rxp"
Jul 09 13:59:20.685515 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.685451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 13:59:20.688090 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.688071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 13:59:20.695880 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:20.695862    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 09 13:59:20.695952 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:20.695942    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs podName:60bf2bd8-1417-48bb-93cd-c7e4763d6817 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:24.695920884 +0000 UTC m=+130.237187659 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs") pod "network-metrics-daemon-nzsrt" (UID: "60bf2bd8-1417-48bb-93cd-c7e4763d6817") : secret "metrics-daemon-secret" not found
Jul 09 13:59:20.786844 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.786792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:59:20.789459 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.789440    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 13:59:20.800661 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.800625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/05625673-1f39-4a55-9a8a-0c55f5245758-original-pull-secret\") pod \"global-pull-secret-syncer-x5q8v\" (UID: \"05625673-1f39-4a55-9a8a-0c55f5245758\") " pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:59:20.887678 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.887631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:59:20.890744 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.890718    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 13:59:20.900635 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.900614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 13:59:20.912239 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:20.912207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ts48\" (UniqueName: \"kubernetes.io/projected/74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6-kube-api-access-5ts48\") pod \"network-check-target-pdv74\" (UID: \"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6\") " pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:59:21.065579 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.065549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x5q8v"
Jul 09 13:59:21.167717 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.167688    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2p5t2\""
Jul 09 13:59:21.175585 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.175541    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:59:21.193807 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.193778    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x5q8v"]
Jul 09 13:59:21.197506 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:21.197478    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05625673_1f39_4a55_9a8a_0c55f5245758.slice/crio-01d718bfd179788859776c4a125ddba3f6171e9a1ae847742cf91cfc7dc65b49 WatchSource:0}: Error finding container 01d718bfd179788859776c4a125ddba3f6171e9a1ae847742cf91cfc7dc65b49: Status 404 returned error can't find the container with id 01d718bfd179788859776c4a125ddba3f6171e9a1ae847742cf91cfc7dc65b49
Jul 09 13:59:21.311864 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.311823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pdv74"]
Jul 09 13:59:21.315705 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:21.315638    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74e2dbe1_a2fe_43cc_9a2e_1e6ad15771f6.slice/crio-402255a28507eafb2a4789effec6648a8975ccb5c2945a17df7c5e6d5b4b1ccf WatchSource:0}: Error finding container 402255a28507eafb2a4789effec6648a8975ccb5c2945a17df7c5e6d5b4b1ccf: Status 404 returned error can't find the container with id 402255a28507eafb2a4789effec6648a8975ccb5c2945a17df7c5e6d5b4b1ccf
Jul 09 13:59:21.450980 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.450939    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5q8v" event={"ID":"05625673-1f39-4a55-9a8a-0c55f5245758","Type":"ContainerStarted","Data":"01d718bfd179788859776c4a125ddba3f6171e9a1ae847742cf91cfc7dc65b49"}
Jul 09 13:59:21.452123 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:21.452097    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdv74" event={"ID":"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6","Type":"ContainerStarted","Data":"402255a28507eafb2a4789effec6648a8975ccb5c2945a17df7c5e6d5b4b1ccf"}
Jul 09 13:59:22.199249 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:22.199212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:22.199717 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:22.199280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:22.199717 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:22.199427    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 13:59:22.199717 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:22.199487    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls podName:855db753-1b10-4049-a83b-e9b1b989775a nodeName:}" failed. No retries permitted until 2026-07-09 13:59:54.199468353 +0000 UTC m=+99.740735121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls") pod "dns-default-xlsp9" (UID: "855db753-1b10-4049-a83b-e9b1b989775a") : secret "dns-default-metrics-tls" not found
Jul 09 13:59:22.199994 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:22.199907    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 13:59:22.199994 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:22.199925    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d68668684-nj8nx: secret "image-registry-tls" not found
Jul 09 13:59:22.199994 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:22.199967    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls podName:8bb59548-7267-4d29-8ba0-617702819a0c nodeName:}" failed. No retries permitted until 2026-07-09 13:59:54.199953196 +0000 UTC m=+99.741219976 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls") pod "image-registry-6d68668684-nj8nx" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c") : secret "image-registry-tls" not found
Jul 09 13:59:26.468040 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:26.467947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x5q8v" event={"ID":"05625673-1f39-4a55-9a8a-0c55f5245758","Type":"ContainerStarted","Data":"d95631edb5d7f6b7a61501783486a7e4b35158060e77c970293808b397fe3d29"}
Jul 09 13:59:26.469513 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:26.469490    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pdv74" event={"ID":"74e2dbe1-a2fe-43cc-9a2e-1e6ad15771f6","Type":"ContainerStarted","Data":"396f73eddc69d3ce5ae22ad09bb4fbc36da434801c67f266f89684dc1e1c8ef2"}
Jul 09 13:59:26.469643 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:26.469629    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:59:26.481484 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:26.481437    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x5q8v" podStartSLOduration=66.608693474 podStartE2EDuration="1m11.48142121s" totalImagesPullingTime="4.872727736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:59:26.480416028 +0000 UTC m=+72.021682810" watchObservedRunningTime="2026-07-09 13:59:26.48142121 +0000 UTC m=+72.022687993"
Jul 09 13:59:26.493991 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:26.493947    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pdv74" podStartSLOduration=66.743258623 podStartE2EDuration="1m11.493931147s" totalImagesPullingTime="4.750672524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:59:26.493042387 +0000 UTC m=+72.034309170" watchObservedRunningTime="2026-07-09 13:59:26.493931147 +0000 UTC m=+72.035197929"
Jul 09 13:59:30.130435 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.130394    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-nw7vv"]
Jul 09 13:59:30.135875 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.135856    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.139096 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.139071    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 13:59:30.139527 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.139509    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 13:59:30.139623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.139542    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 13:59:30.139623 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.139606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 13:59:30.140032 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.140015    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cvk42\""
Jul 09 13:59:30.153292 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.153260    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nw7vv"]
Jul 09 13:59:30.263420 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.263377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h2gw\" (UniqueName: \"kubernetes.io/projected/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-api-access-9h2gw\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.263667 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.263447    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47cfd0c9-380d-4553-815f-ecfa50a3e591-crio-socket\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.263667 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.263475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.263667 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.263533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47cfd0c9-380d-4553-815f-ecfa50a3e591-data-volume\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.263667 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.263616    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364054 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h2gw\" (UniqueName: \"kubernetes.io/projected/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-api-access-9h2gw\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364251 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47cfd0c9-380d-4553-815f-ecfa50a3e591-crio-socket\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364251 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364147    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/47cfd0c9-380d-4553-815f-ecfa50a3e591-crio-socket\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364251 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364398 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47cfd0c9-380d-4553-815f-ecfa50a3e591-data-volume\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364398 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.364398 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.364358    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:30.364545 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.364408    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 13:59:30.364545 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.364411    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:30.864396974 +0000 UTC m=+76.405663735 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:30.364545 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.364494    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:30.864476586 +0000 UTC m=+76.405743347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : configmap "kube-rbac-proxy" not found
Jul 09 13:59:30.364697 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.364630    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/47cfd0c9-380d-4553-815f-ecfa50a3e591-data-volume\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.373605 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.373553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h2gw\" (UniqueName: \"kubernetes.io/projected/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-api-access-9h2gw\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.869310 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.869250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.869551 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:30.869342    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:30.869551 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.869418    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 13:59:30.869551 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.869446    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:30.869551 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.869472    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:31.869457587 +0000 UTC m=+77.410724356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : configmap "kube-rbac-proxy" not found
Jul 09 13:59:30.869551 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:30.869513    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:31.869494412 +0000 UTC m=+77.410761200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:31.878100 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:31.878038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:31.878644 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:31.878115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:31.878644 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:31.878201    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 13:59:31.878644 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:31.878215    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:31.878644 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:31.878266    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:33.878250839 +0000 UTC m=+79.419517599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : configmap "kube-rbac-proxy" not found
Jul 09 13:59:31.878644 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:31.878280    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:33.878273736 +0000 UTC m=+79.419540497 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:31.930018 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:31.929990    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dxqhw_42df1555-d3fb-4246-807a-e18a880a86a5/dns-node-resolver/0.log"
Jul 09 13:59:33.130105 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:33.130078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkbvh_57169df3-14c5-4df9-990d-77581fa9941f/node-ca/0.log"
Jul 09 13:59:33.894110 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:33.894061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:33.894301 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:33.894146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:33.894301 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:33.894194    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 13:59:33.894301 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:33.894256    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:33.894301 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:33.894269    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:37.894250009 +0000 UTC m=+83.435516772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : configmap "kube-rbac-proxy" not found
Jul 09 13:59:33.894301 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:33.894297    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:37.894284853 +0000 UTC m=+83.435551614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:33.930014 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:33.929984    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t6xq5_1e20d41c-0a27-40b7-a77f-3b3d88e9979a/serve-healthcheck-canary/0.log"
Jul 09 13:59:34.667889 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.667845    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-qskv9"]
Jul 09 13:59:34.669636 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.669620    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.671795 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.671772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 09 13:59:34.671929 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.671823    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 09 13:59:34.672051 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.672036    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 09 13:59:34.672796 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.672776    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-tlmvn\""
Jul 09 13:59:34.673396 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.673377    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 09 13:59:34.676296 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.676266    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 09 13:59:34.680369 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.680346    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-qskv9"]
Jul 09 13:59:34.800477 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.800434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-config\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.800684 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.800491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-trusted-ca\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.800684 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.800515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfhdl\" (UniqueName: \"kubernetes.io/projected/f8bca19a-836b-4f10-9d32-883deea28646-kube-api-access-bfhdl\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.800684 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.800639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8bca19a-836b-4f10-9d32-883deea28646-serving-cert\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.901743 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.901706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8bca19a-836b-4f10-9d32-883deea28646-serving-cert\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.901878 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.901797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-config\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.901878 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.901834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-trusted-ca\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.901878 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.901858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfhdl\" (UniqueName: \"kubernetes.io/projected/f8bca19a-836b-4f10-9d32-883deea28646-kube-api-access-bfhdl\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.902519 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.902491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-config\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.902806 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.902783    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8bca19a-836b-4f10-9d32-883deea28646-trusted-ca\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.904374 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.904357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f8bca19a-836b-4f10-9d32-883deea28646-serving-cert\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.911775 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.911743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfhdl\" (UniqueName: \"kubernetes.io/projected/f8bca19a-836b-4f10-9d32-883deea28646-kube-api-access-bfhdl\") pod \"console-operator-7866497494-qskv9\" (UID: \"f8bca19a-836b-4f10-9d32-883deea28646\") " pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:34.981001 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:34.980900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:35.126052 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:35.126012    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-qskv9"]
Jul 09 13:59:35.132367 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:35.132335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8bca19a_836b_4f10_9d32_883deea28646.slice/crio-f3a635f57c2b64d4be27372529344d1dc5f45bd1572af233716b99202bc70e8d WatchSource:0}: Error finding container f3a635f57c2b64d4be27372529344d1dc5f45bd1572af233716b99202bc70e8d: Status 404 returned error can't find the container with id f3a635f57c2b64d4be27372529344d1dc5f45bd1572af233716b99202bc70e8d
Jul 09 13:59:35.495466 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:35.495427    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qskv9" event={"ID":"f8bca19a-836b-4f10-9d32-883deea28646","Type":"ContainerStarted","Data":"f3a635f57c2b64d4be27372529344d1dc5f45bd1572af233716b99202bc70e8d"}
Jul 09 13:59:37.501992 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.501961    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/0.log"
Jul 09 13:59:37.502456 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.502006    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8bca19a-836b-4f10-9d32-883deea28646" containerID="85978a6f2c6b257d0c0e931e45ea19ca94ece9f94322a5614fba0e48e0aed29b" exitCode=255
Jul 09 13:59:37.502456 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.502046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qskv9" event={"ID":"f8bca19a-836b-4f10-9d32-883deea28646","Type":"ContainerDied","Data":"85978a6f2c6b257d0c0e931e45ea19ca94ece9f94322a5614fba0e48e0aed29b"}
Jul 09 13:59:37.502456 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.502356    2580 scope.go:117] "RemoveContainer" containerID="85978a6f2c6b257d0c0e931e45ea19ca94ece9f94322a5614fba0e48e0aed29b"
Jul 09 13:59:37.925231 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.925134    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:37.925231 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:37.925212    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:37.925431 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:37.925250    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 13:59:37.925431 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:37.925304    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:37.925431 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:37.925322    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:45.925306189 +0000 UTC m=+91.466572950 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : configmap "kube-rbac-proxy" not found
Jul 09 13:59:37.925431 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:37.925343    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 13:59:45.925332436 +0000 UTC m=+91.466599197 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:38.506126 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506100    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/1.log"
Jul 09 13:59:38.506589 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506488    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/0.log"
Jul 09 13:59:38.506589 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506530    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8bca19a-836b-4f10-9d32-883deea28646" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c" exitCode=255
Jul 09 13:59:38.506696 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qskv9" event={"ID":"f8bca19a-836b-4f10-9d32-883deea28646","Type":"ContainerDied","Data":"6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"}
Jul 09 13:59:38.506696 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506672    2580 scope.go:117] "RemoveContainer" containerID="85978a6f2c6b257d0c0e931e45ea19ca94ece9f94322a5614fba0e48e0aed29b"
Jul 09 13:59:38.506879 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:38.506863    2580 scope.go:117] "RemoveContainer" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"
Jul 09 13:59:38.507063 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:38.507045    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 13:59:39.510744 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:39.510708    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/1.log"
Jul 09 13:59:39.511136 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:39.511028    2580 scope.go:117] "RemoveContainer" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"
Jul 09 13:59:39.511199 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:39.511169    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 13:59:41.260794 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.260760    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rcbk7"]
Jul 09 13:59:41.264820 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.264804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.267458 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.267440    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-m64gg\""
Jul 09 13:59:41.267584 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.267441    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 09 13:59:41.268514 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.268493    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 09 13:59:41.268737 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.268720    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 09 13:59:41.268837 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.268779    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 09 13:59:41.271618 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.271477    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rcbk7"]
Jul 09 13:59:41.353593 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.353534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-key\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.353794 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.353610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-cabundle\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.353794 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.353704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnkbk\" (UniqueName: \"kubernetes.io/projected/6a45bf90-ab78-45fc-9e14-3b5693c59977-kube-api-access-lnkbk\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.454996 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.454951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnkbk\" (UniqueName: \"kubernetes.io/projected/6a45bf90-ab78-45fc-9e14-3b5693c59977-kube-api-access-lnkbk\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.455183 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.455012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-key\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.455183 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.455063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-cabundle\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.455807 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.455782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-cabundle\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.457519 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.457496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/6a45bf90-ab78-45fc-9e14-3b5693c59977-signing-key\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.464624 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.464597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnkbk\" (UniqueName: \"kubernetes.io/projected/6a45bf90-ab78-45fc-9e14-3b5693c59977-kube-api-access-lnkbk\") pod \"service-ca-79f68b9f4f-rcbk7\" (UID: \"6a45bf90-ab78-45fc-9e14-3b5693c59977\") " pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.574267 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.574211    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7"
Jul 09 13:59:41.712078 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:41.711907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-rcbk7"]
Jul 09 13:59:41.714494 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:41.714457    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a45bf90_ab78_45fc_9e14_3b5693c59977.slice/crio-2f84d85cf0a3bdd5077298ac1951a2abf9fd13fe0082691ba87adc2c7941b382 WatchSource:0}: Error finding container 2f84d85cf0a3bdd5077298ac1951a2abf9fd13fe0082691ba87adc2c7941b382: Status 404 returned error can't find the container with id 2f84d85cf0a3bdd5077298ac1951a2abf9fd13fe0082691ba87adc2c7941b382
Jul 09 13:59:42.519671 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:42.519633    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7" event={"ID":"6a45bf90-ab78-45fc-9e14-3b5693c59977","Type":"ContainerStarted","Data":"2f84d85cf0a3bdd5077298ac1951a2abf9fd13fe0082691ba87adc2c7941b382"}
Jul 09 13:59:44.531864 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:44.531831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7" event={"ID":"6a45bf90-ab78-45fc-9e14-3b5693c59977","Type":"ContainerStarted","Data":"98d6c3ec88fe3ae74b48b8bd08c400ae77ee30a54b28b5020989a2f51fb769b3"}
Jul 09 13:59:44.548130 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:44.548075    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-rcbk7" podStartSLOduration=1.64016399 podStartE2EDuration="3.548058133s" totalImagesPullingTime="1.907894143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:59:44.547805984 +0000 UTC m=+90.089072778" watchObservedRunningTime="2026-07-09 13:59:44.548058133 +0000 UTC m=+90.089324917"
Jul 09 13:59:44.982043 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:44.981996    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:44.982228 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:44.982073    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 13:59:44.982542 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:44.982526    2580 scope.go:117] "RemoveContainer" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"
Jul 09 13:59:44.982778 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:44.982754    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 13:59:45.992805 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:45.992755    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:45.993295 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:45.992824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:45.993295 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:45.992949    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:45.993295 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:45.993032    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls podName:47cfd0c9-380d-4553-815f-ecfa50a3e591 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:01.993010512 +0000 UTC m=+107.534277278 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls") pod "insights-runtime-extractor-nw7vv" (UID: "47cfd0c9-380d-4553-815f-ecfa50a3e591") : secret "insights-runtime-extractor-tls" not found
Jul 09 13:59:45.993459 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:45.993316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/47cfd0c9-380d-4553-815f-ecfa50a3e591-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 13:59:54.259394 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.259336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:54.259394 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.259409    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:54.261894 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.261869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/855db753-1b10-4049-a83b-e9b1b989775a-metrics-tls\") pod \"dns-default-xlsp9\" (UID: \"855db753-1b10-4049-a83b-e9b1b989775a\") " pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:54.261994 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.261957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"image-registry-6d68668684-nj8nx\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") " pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:54.514825 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.514734    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-n8xtq\""
Jul 09 13:59:54.522683 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.522657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:54.528541 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.528517    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hf6lw\""
Jul 09 13:59:54.536475 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.536449    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:54.670262 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.670230    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 13:59:54.674266 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:54.674240    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bb59548_7267_4d29_8ba0_617702819a0c.slice/crio-5ce6e2222b1c19484a122d79b48183b029ef0b7c5090eb25958b86f83e0f3950 WatchSource:0}: Error finding container 5ce6e2222b1c19484a122d79b48183b029ef0b7c5090eb25958b86f83e0f3950: Status 404 returned error can't find the container with id 5ce6e2222b1c19484a122d79b48183b029ef0b7c5090eb25958b86f83e0f3950
Jul 09 13:59:54.682477 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:54.682458    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xlsp9"]
Jul 09 13:59:54.686510 ip-10-0-143-235 kubenswrapper[2580]: W0709 13:59:54.686479    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod855db753_1b10_4049_a83b_e9b1b989775a.slice/crio-8d1a7bca65f54c72e1ca97f3b35a5042162c1868e33e8951debd20efa7181f95 WatchSource:0}: Error finding container 8d1a7bca65f54c72e1ca97f3b35a5042162c1868e33e8951debd20efa7181f95: Status 404 returned error can't find the container with id 8d1a7bca65f54c72e1ca97f3b35a5042162c1868e33e8951debd20efa7181f95
Jul 09 13:59:55.571730 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:55.571691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" event={"ID":"8bb59548-7267-4d29-8ba0-617702819a0c","Type":"ContainerStarted","Data":"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"}
Jul 09 13:59:55.572283 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:55.572253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" event={"ID":"8bb59548-7267-4d29-8ba0-617702819a0c","Type":"ContainerStarted","Data":"5ce6e2222b1c19484a122d79b48183b029ef0b7c5090eb25958b86f83e0f3950"}
Jul 09 13:59:55.572411 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:55.572297    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 13:59:55.573256 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:55.573232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlsp9" event={"ID":"855db753-1b10-4049-a83b-e9b1b989775a","Type":"ContainerStarted","Data":"8d1a7bca65f54c72e1ca97f3b35a5042162c1868e33e8951debd20efa7181f95"}
Jul 09 13:59:55.596775 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:55.596719    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" podStartSLOduration=99.596704592 podStartE2EDuration="1m39.596704592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:59:55.594934962 +0000 UTC m=+101.136201767" watchObservedRunningTime="2026-07-09 13:59:55.596704592 +0000 UTC m=+101.137971375"
Jul 09 13:59:57.475154 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:57.475096    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pdv74"
Jul 09 13:59:57.581720 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:57.581666    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlsp9" event={"ID":"855db753-1b10-4049-a83b-e9b1b989775a","Type":"ContainerStarted","Data":"2d019bd414580c36ca36561c06a506d93a1a5606f04ffe3cc9a2b4f58ffc406f"}
Jul 09 13:59:57.581919 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:57.581740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xlsp9" event={"ID":"855db753-1b10-4049-a83b-e9b1b989775a","Type":"ContainerStarted","Data":"a6898d5c655e83f94b9f303d9fa6c90228bb8630dab48e1bf1ecd274f8322d24"}
Jul 09 13:59:58.054712 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.054674    2580 scope.go:117] "RemoveContainer" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"
Jul 09 13:59:58.586067 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586038    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 13:59:58.586478 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/1.log"
Jul 09 13:59:58.586478 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586440    2580 generic.go:356] "Generic (PLEG): container finished" podID="f8bca19a-836b-4f10-9d32-883deea28646" containerID="bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a" exitCode=255
Jul 09 13:59:58.586549 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qskv9" event={"ID":"f8bca19a-836b-4f10-9d32-883deea28646","Type":"ContainerDied","Data":"bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a"}
Jul 09 13:59:58.586608 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586592    2580 scope.go:117] "RemoveContainer" containerID="6257cb710164c5a01dedcfb84a6f030bd9886b780e6e4d37ef7fd84dda68590c"
Jul 09 13:59:58.586710 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586694    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xlsp9"
Jul 09 13:59:58.586953 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.586933    2580 scope.go:117] "RemoveContainer" containerID="bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a"
Jul 09 13:59:58.587116 ip-10-0-143-235 kubenswrapper[2580]: E0709 13:59:58.587096    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 13:59:58.603236 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:58.603193    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xlsp9" podStartSLOduration=66.07935084 podStartE2EDuration="1m8.603176988s" totalImagesPullingTime="2.523826148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 13:59:57.598347675 +0000 UTC m=+103.139614479" watchObservedRunningTime="2026-07-09 13:59:58.603176988 +0000 UTC m=+104.144443773"
Jul 09 13:59:59.591287 ip-10-0-143-235 kubenswrapper[2580]: I0709 13:59:59.591225    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:00:00.373210 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.373179    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"]
Jul 09 14:00:00.375799 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.375776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.378177 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.378154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 14:00:00.378398 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.378383    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 14:00:00.378694 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.378680    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 14:00:00.379496 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.379480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 14:00:00.391968 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.391944    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"]
Jul 09 14:00:00.483550 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.483515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"]
Jul 09 14:00:00.485488 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.485469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:00.488165 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.488134    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dp2j9\""
Jul 09 14:00:00.488300 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.488194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 09 14:00:00.498009 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.497987    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"]
Jul 09 14:00:00.509018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.508993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7611484d-0a02-4689-acad-07b9b835633c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.509146 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.509054    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66sqg\" (UniqueName: \"kubernetes.io/projected/7611484d-0a02-4689-acad-07b9b835633c-kube-api-access-66sqg\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.509146 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.509079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7611484d-0a02-4689-acad-07b9b835633c-tmp\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.610056 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.610021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43c65f41-f0bd-43e3-bd66-2a121bfece86-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9bp7q\" (UID: \"43c65f41-f0bd-43e3-bd66-2a121bfece86\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:00.610488 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.610085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66sqg\" (UniqueName: \"kubernetes.io/projected/7611484d-0a02-4689-acad-07b9b835633c-kube-api-access-66sqg\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.610488 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.610148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7611484d-0a02-4689-acad-07b9b835633c-tmp\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.610488 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.610201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7611484d-0a02-4689-acad-07b9b835633c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.610709 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.610658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7611484d-0a02-4689-acad-07b9b835633c-tmp\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.612744 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.612726    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/7611484d-0a02-4689-acad-07b9b835633c-klusterlet-config\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.624644 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.624559    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66sqg\" (UniqueName: \"kubernetes.io/projected/7611484d-0a02-4689-acad-07b9b835633c-kube-api-access-66sqg\") pod \"klusterlet-addon-workmgr-7dbc59bf7b-d6tvz\" (UID: \"7611484d-0a02-4689-acad-07b9b835633c\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.685281 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.685242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:00.711685 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.711647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43c65f41-f0bd-43e3-bd66-2a121bfece86-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9bp7q\" (UID: \"43c65f41-f0bd-43e3-bd66-2a121bfece86\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:00.714255 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.714228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/43c65f41-f0bd-43e3-bd66-2a121bfece86-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-9bp7q\" (UID: \"43c65f41-f0bd-43e3-bd66-2a121bfece86\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:00.794894 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.794854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:00.818518 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.818491    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"]
Jul 09 14:00:00.821203 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:00.821183    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7611484d_0a02_4689_acad_07b9b835633c.slice/crio-bc0a75cdc2ca554681bbb8562c061e985f01581547995785ab5b067cb0e3c99b WatchSource:0}: Error finding container bc0a75cdc2ca554681bbb8562c061e985f01581547995785ab5b067cb0e3c99b: Status 404 returned error can't find the container with id bc0a75cdc2ca554681bbb8562c061e985f01581547995785ab5b067cb0e3c99b
Jul 09 14:00:00.924801 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:00.924779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"]
Jul 09 14:00:00.927245 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:00.927213    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43c65f41_f0bd_43e3_bd66_2a121bfece86.slice/crio-e6b85e46745c906a35fcfd12e562088b28ee0fb5398f1f29970602c05e4a2118 WatchSource:0}: Error finding container e6b85e46745c906a35fcfd12e562088b28ee0fb5398f1f29970602c05e4a2118: Status 404 returned error can't find the container with id e6b85e46745c906a35fcfd12e562088b28ee0fb5398f1f29970602c05e4a2118
Jul 09 14:00:01.598615 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:01.598580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q" event={"ID":"43c65f41-f0bd-43e3-bd66-2a121bfece86","Type":"ContainerStarted","Data":"e6b85e46745c906a35fcfd12e562088b28ee0fb5398f1f29970602c05e4a2118"}
Jul 09 14:00:01.599610 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:01.599589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz" event={"ID":"7611484d-0a02-4689-acad-07b9b835633c","Type":"ContainerStarted","Data":"bc0a75cdc2ca554681bbb8562c061e985f01581547995785ab5b067cb0e3c99b"}
Jul 09 14:00:02.020865 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:02.020763    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 14:00:02.023246 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:02.023227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/47cfd0c9-380d-4553-815f-ecfa50a3e591-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-nw7vv\" (UID: \"47cfd0c9-380d-4553-815f-ecfa50a3e591\") " pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 14:00:02.245363 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:02.245327    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-nw7vv"
Jul 09 14:00:02.392723 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:02.392689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-nw7vv"]
Jul 09 14:00:02.618997 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:02.618958    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47cfd0c9_380d_4553_815f_ecfa50a3e591.slice/crio-9a772ee6c61693815ec45662e0c584b54dfb4e43e25eced3deaab23cf248690c WatchSource:0}: Error finding container 9a772ee6c61693815ec45662e0c584b54dfb4e43e25eced3deaab23cf248690c: Status 404 returned error can't find the container with id 9a772ee6c61693815ec45662e0c584b54dfb4e43e25eced3deaab23cf248690c
Jul 09 14:00:03.609048 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:03.609002    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nw7vv" event={"ID":"47cfd0c9-380d-4553-815f-ecfa50a3e591","Type":"ContainerStarted","Data":"9a772ee6c61693815ec45662e0c584b54dfb4e43e25eced3deaab23cf248690c"}
Jul 09 14:00:03.611074 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:03.611043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q" event={"ID":"43c65f41-f0bd-43e3-bd66-2a121bfece86","Type":"ContainerStarted","Data":"a48183e4499ea17bf8aa2c8fbf68913dff75bff6bac9976bffc0c02e9d52d33c"}
Jul 09 14:00:03.611279 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:03.611257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:03.617926 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:03.617904    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q"
Jul 09 14:00:03.626658 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:03.626600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-9bp7q" podStartSLOduration=1.911058996 podStartE2EDuration="3.626559204s" totalImagesPullingTime="1.715500208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:03.6255367 +0000 UTC m=+109.166803510" watchObservedRunningTime="2026-07-09 14:00:03.626559204 +0000 UTC m=+109.167825988"
Jul 09 14:00:04.981313 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:04.981275    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 14:00:04.981720 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:04.981336    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 14:00:04.981720 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:04.981707    2580 scope.go:117] "RemoveContainer" containerID="bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a"
Jul 09 14:00:04.981887 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:04.981865    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 14:00:08.594328 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:08.594294    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xlsp9"
Jul 09 14:00:09.634313 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:09.634281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz" event={"ID":"7611484d-0a02-4689-acad-07b9b835633c","Type":"ContainerStarted","Data":"99ebbba7a1c8dab59bf58d5f24c7446a6ce0d58d5a23b6be4bf8bb1cfff06fbf"}
Jul 09 14:00:09.634795 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:09.634480    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:09.636099 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:09.636063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nw7vv" event={"ID":"47cfd0c9-380d-4553-815f-ecfa50a3e591","Type":"ContainerStarted","Data":"74261c1da7c1c67b0730db5c7e2321f006fa3d79996765d1201ffd9be8ee27ce"}
Jul 09 14:00:09.636284 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:09.636258    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz"
Jul 09 14:00:09.650065 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:09.650013    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-7dbc59bf7b-d6tvz" podStartSLOduration=1.32307533 podStartE2EDuration="9.649999292s" totalImagesPullingTime="8.326923962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:09.649222707 +0000 UTC m=+115.190489492" watchObservedRunningTime="2026-07-09 14:00:09.649999292 +0000 UTC m=+115.191266075"
Jul 09 14:00:10.641345 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:10.641303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nw7vv" event={"ID":"47cfd0c9-380d-4553-815f-ecfa50a3e591","Type":"ContainerStarted","Data":"62349370b308a72e20e9f11360fdc6731eb2cf33cfa3d5b3c85c5e2cfed17cc3"}
Jul 09 14:00:12.652031 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.651995    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"]
Jul 09 14:00:12.668394 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.668352    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"]
Jul 09 14:00:12.668562 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.668491    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.671797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.671393    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 14:00:12.671797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.671649    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 09 14:00:12.671797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.671792    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 09 14:00:12.678456 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.678189    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-cmgb4\""
Jul 09 14:00:12.678456 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.678268    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 14:00:12.678684 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.678635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 14:00:12.682522 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.682262    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"]
Jul 09 14:00:12.689022 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.689001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.692349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.692323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 09 14:00:12.692771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.692752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 09 14:00:12.692991 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.692976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 09 14:00:12.693186 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.693165    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"]
Jul 09 14:00:12.693474 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.693455    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-64cc7\""
Jul 09 14:00:12.703489 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.703463    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pdms9"]
Jul 09 14:00:12.705294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.705509 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.705620 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.705620 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.705620 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705605    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.705783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxv27\" (UniqueName: \"kubernetes.io/projected/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-kube-api-access-zxv27\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.705783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.705783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.705922 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f10aad93-e163-479b-a921-47331e292259-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.705922 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.705849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqpw\" (UniqueName: \"kubernetes.io/projected/f10aad93-e163-479b-a921-47331e292259-kube-api-access-pqqpw\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.707269 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.707244    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.709758 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.709693    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 14:00:12.709878 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.709856    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ndd5r\""
Jul 09 14:00:12.711193 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.711173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 14:00:12.711281 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.711206    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 14:00:12.806599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxv27\" (UniqueName: \"kubernetes.io/projected/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-kube-api-access-zxv27\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.806777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-textfile\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.806777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806658    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.806777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.806777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806730    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.806777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f10aad93-e163-479b-a921-47331e292259-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-root\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806847    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-metrics-client-ca\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806869    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqpw\" (UniqueName: \"kubernetes.io/projected/f10aad93-e163-479b-a921-47331e292259-kube-api-access-pqqpw\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.806983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807041 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-sys\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807078    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-wtmp\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q9p6\" (UniqueName: \"kubernetes.io/projected/73720c1c-9f25-4f26-b17a-19f5f75ed420-kube-api-access-6q9p6\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.807377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f10aad93-e163-479b-a921-47331e292259-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.807721 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807378    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.807919 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.807872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.808232 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.808150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.809997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.809976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.809997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.809986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.810356 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.810334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f10aad93-e163-479b-a921-47331e292259-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.810465 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.810449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.814294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.814267    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqpw\" (UniqueName: \"kubernetes.io/projected/f10aad93-e163-479b-a921-47331e292259-kube-api-access-pqqpw\") pod \"kube-state-metrics-785778ff74-dlgnx\" (UID: \"f10aad93-e163-479b-a921-47331e292259\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:12.814691 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.814671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxv27\" (UniqueName: \"kubernetes.io/projected/2608b02b-2b5b-4bd2-a85f-3a999d3bd449-kube-api-access-zxv27\") pod \"openshift-state-metrics-84bd68b896-9xddl\" (UID: \"2608b02b-2b5b-4bd2-a85f-3a999d3bd449\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:12.907830 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-root\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.907830 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.907830 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-metrics-client-ca\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-root\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-sys\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:12.907970    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.907977    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-wtmp\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q9p6\" (UniqueName: \"kubernetes.io/projected/73720c1c-9f25-4f26-b17a-19f5f75ed420-kube-api-access-6q9p6\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:12.908036    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls podName:73720c1c-9f25-4f26-b17a-19f5f75ed420 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:13.408014014 +0000 UTC m=+118.949280781 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls") pod "node-exporter-pdms9" (UID: "73720c1c-9f25-4f26-b17a-19f5f75ed420") : secret "node-exporter-tls" not found
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-sys\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-textfile\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-wtmp\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908450    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-textfile\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-accelerators-collector-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.908859 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.908838    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73720c1c-9f25-4f26-b17a-19f5f75ed420-metrics-client-ca\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.911044 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.911022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.916917 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.916885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q9p6\" (UniqueName: \"kubernetes.io/projected/73720c1c-9f25-4f26-b17a-19f5f75ed420-kube-api-access-6q9p6\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:12.987745 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:12.987702    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"
Jul 09 14:00:13.007162 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.007134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"
Jul 09 14:00:13.145976 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.145913    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl"]
Jul 09 14:00:13.149133 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:13.149090    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2608b02b_2b5b_4bd2_a85f_3a999d3bd449.slice/crio-4157e6cfa31bfd3403953bb66170eee07cb3ea495d311a4f7c4b7ae531ceb370 WatchSource:0}: Error finding container 4157e6cfa31bfd3403953bb66170eee07cb3ea495d311a4f7c4b7ae531ceb370: Status 404 returned error can't find the container with id 4157e6cfa31bfd3403953bb66170eee07cb3ea495d311a4f7c4b7ae531ceb370
Jul 09 14:00:13.164483 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.164456    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlgnx"]
Jul 09 14:00:13.168161 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:13.168132    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf10aad93_e163_479b_a921_47331e292259.slice/crio-9ba15c0e4623ad957fb3baae41be2afc4f1e13a0776a453b582cff5039e7a802 WatchSource:0}: Error finding container 9ba15c0e4623ad957fb3baae41be2afc4f1e13a0776a453b582cff5039e7a802: Status 404 returned error can't find the container with id 9ba15c0e4623ad957fb3baae41be2afc4f1e13a0776a453b582cff5039e7a802
Jul 09 14:00:13.412652 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.412600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:13.412860 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:13.412737    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 09 14:00:13.412860 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:13.412809    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls podName:73720c1c-9f25-4f26-b17a-19f5f75ed420 nodeName:}" failed. No retries permitted until 2026-07-09 14:00:14.412789889 +0000 UTC m=+119.954056651 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls") pod "node-exporter-pdms9" (UID: "73720c1c-9f25-4f26-b17a-19f5f75ed420") : secret "node-exporter-tls" not found
Jul 09 14:00:13.654517 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.654463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-nw7vv" event={"ID":"47cfd0c9-380d-4553-815f-ecfa50a3e591","Type":"ContainerStarted","Data":"60bc0845915f893d1caa5ef5d225581444816b957b21d393bc96fd595d18472e"}
Jul 09 14:00:13.655805 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.655777    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx" event={"ID":"f10aad93-e163-479b-a921-47331e292259","Type":"ContainerStarted","Data":"9ba15c0e4623ad957fb3baae41be2afc4f1e13a0776a453b582cff5039e7a802"}
Jul 09 14:00:13.657744 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.657714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl" event={"ID":"2608b02b-2b5b-4bd2-a85f-3a999d3bd449","Type":"ContainerStarted","Data":"9a9568c328ea48a792ace2a2df71e412eecf15adccb20eaa2b4f7f8bc3d350ba"}
Jul 09 14:00:13.657841 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.657765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl" event={"ID":"2608b02b-2b5b-4bd2-a85f-3a999d3bd449","Type":"ContainerStarted","Data":"15cd21d193f0780b6b2bf94a78c2314664af34b676359781df006b1241e5a065"}
Jul 09 14:00:13.657841 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.657786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl" event={"ID":"2608b02b-2b5b-4bd2-a85f-3a999d3bd449","Type":"ContainerStarted","Data":"4157e6cfa31bfd3403953bb66170eee07cb3ea495d311a4f7c4b7ae531ceb370"}
Jul 09 14:00:13.672611 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:13.672536    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-nw7vv" podStartSLOduration=33.56029851 podStartE2EDuration="43.672519377s" totalImagesPullingTime="10.112220867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 13:59:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:13.671399734 +0000 UTC m=+119.212666557" watchObservedRunningTime="2026-07-09 14:00:13.672519377 +0000 UTC m=+119.213786159"
Jul 09 14:00:14.422074 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.422033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:14.424386 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.424359    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/73720c1c-9f25-4f26-b17a-19f5f75ed420-node-exporter-tls\") pod \"node-exporter-pdms9\" (UID: \"73720c1c-9f25-4f26-b17a-19f5f75ed420\") " pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:14.519274 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.519236    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pdms9"
Jul 09 14:00:14.528767 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.528606    2580 patch_prober.go:28] interesting pod/image-registry-6d68668684-nj8nx container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 14:00:14.528767 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.528673    2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 14:00:14.532051 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:14.532018    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73720c1c_9f25_4f26_b17a_19f5f75ed420.slice/crio-ba41d84cc39542af0a95b25390e8195a68117ff413b3aceede8b991d31dd4d5d WatchSource:0}: Error finding container ba41d84cc39542af0a95b25390e8195a68117ff413b3aceede8b991d31dd4d5d: Status 404 returned error can't find the container with id ba41d84cc39542af0a95b25390e8195a68117ff413b3aceede8b991d31dd4d5d
Jul 09 14:00:14.663148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:14.663109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdms9" event={"ID":"73720c1c-9f25-4f26-b17a-19f5f75ed420","Type":"ContainerStarted","Data":"ba41d84cc39542af0a95b25390e8195a68117ff413b3aceede8b991d31dd4d5d"}
Jul 09 14:00:15.668645 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:15.668607    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx" event={"ID":"f10aad93-e163-479b-a921-47331e292259","Type":"ContainerStarted","Data":"0ac119a2cd436a647a51a9107cdb3bf4f39eebf0c3585fc34d5f51d619c5847c"}
Jul 09 14:00:15.671188 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:15.671159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl" event={"ID":"2608b02b-2b5b-4bd2-a85f-3a999d3bd449","Type":"ContainerStarted","Data":"58871db5c9626cc03e7bbbe34f236d7dfdba313a507bd5b7642f8c412daf0560"}
Jul 09 14:00:15.691727 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:15.691043    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9xddl" podStartSLOduration=2.019774075 podStartE2EDuration="3.691020316s" totalImagesPullingTime="1.671246241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:15.689131958 +0000 UTC m=+121.230398765" watchObservedRunningTime="2026-07-09 14:00:15.691020316 +0000 UTC m=+121.232287102"
Jul 09 14:00:16.055174 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.055148    2580 scope.go:117] "RemoveContainer" containerID="bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a"
Jul 09 14:00:16.055338 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:16.055316    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7866497494-qskv9_openshift-console-operator(f8bca19a-836b-4f10-9d32-883deea28646)\"" pod="openshift-console-operator/console-operator-7866497494-qskv9" podUID="f8bca19a-836b-4f10-9d32-883deea28646"
Jul 09 14:00:16.677018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.676926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx" event={"ID":"f10aad93-e163-479b-a921-47331e292259","Type":"ContainerStarted","Data":"3442ad03d891be957a58647ee291936b939eda76551c7540e0119c03ea0147fa"}
Jul 09 14:00:16.677018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.677014    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx" event={"ID":"f10aad93-e163-479b-a921-47331e292259","Type":"ContainerStarted","Data":"fb44a372d4ca9606e09dd5eb6291bc90542524541563b8cb20334a3fec1648db"}
Jul 09 14:00:16.678687 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.678661    2580 generic.go:356] "Generic (PLEG): container finished" podID="73720c1c-9f25-4f26-b17a-19f5f75ed420" containerID="b525a34136f165286305889339b3e62b29efddaa5763fc2d6588c75f39cf6efe" exitCode=0
Jul 09 14:00:16.678789 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.678720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdms9" event={"ID":"73720c1c-9f25-4f26-b17a-19f5f75ed420","Type":"ContainerDied","Data":"b525a34136f165286305889339b3e62b29efddaa5763fc2d6588c75f39cf6efe"}
Jul 09 14:00:16.696124 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:16.695982    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlgnx" podStartSLOduration=2.273492407 podStartE2EDuration="4.695960173s" totalImagesPullingTime="2.422467766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:16.694401251 +0000 UTC m=+122.235668075" watchObservedRunningTime="2026-07-09 14:00:16.695960173 +0000 UTC m=+122.237226959"
Jul 09 14:00:17.586685 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:17.586653    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 14:00:17.684090 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:17.684054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdms9" event={"ID":"73720c1c-9f25-4f26-b17a-19f5f75ed420","Type":"ContainerStarted","Data":"17388f0f9ad84e4c2eef154b092a8fd05bec0a38856bbd9b251a053acdf3808c"}
Jul 09 14:00:17.684586 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:17.684123    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pdms9" event={"ID":"73720c1c-9f25-4f26-b17a-19f5f75ed420","Type":"ContainerStarted","Data":"bb49fa487a05ef5ce8f045b3a166a62ae760bb0b2949bce4d4af02853ce95e0c"}
Jul 09 14:00:17.705375 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:17.705305    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pdms9" podStartSLOduration=3.830333038 podStartE2EDuration="5.705288418s" totalImagesPullingTime="1.799704166s" totalInitContainerRuntime="75.251214ms" isStatefulPod=true podCreationTimestamp="2026-07-09 14:00:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:17.704235759 +0000 UTC m=+123.245502542" watchObservedRunningTime="2026-07-09 14:00:17.705288418 +0000 UTC m=+123.246555200"
Jul 09 14:00:18.878273 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.878236    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:18.882022 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.882001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.885282 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.885258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 14:00:18.885481 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.885468    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 14:00:18.885853 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.885835    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 14:00:18.886088 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886071    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 14:00:18.886203 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886181    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 14:00:18.886313 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886296    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 14:00:18.886313 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886307    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 14:00:18.886424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886214    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 14:00:18.886424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 14:00:18.886424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4id6ojmm6pv95\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886278    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kpgh9\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886269    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886213    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886311    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 14:00:18.886558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.886406    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 14:00:18.888611 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.888590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 14:00:18.898097 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.898072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:18.962357 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962357 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962481    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962498    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gs9rq\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962689    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962779    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962862    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:18.962966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:18.962923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063712    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.063964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.063981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064096    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gs9rq\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.064726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.064680    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.065226 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.065201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.067842 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.067815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.068034 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.068009    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.068632 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.068607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.068632 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.068610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.068900 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.068877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.068970 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.068881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.069545 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.069093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.069545 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.069498    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.069745 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.069700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070118 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070250 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070100    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070250 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070119    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070428 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070524 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.070703 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.070686    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.071433 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.071411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.073461 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.073444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gs9rq\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq\") pod \"prometheus-k8s-0\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.191468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.191372    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:19.344011 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.343977    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:00:19.347595 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:19.347536    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee375ef9_3c7c_416a_a804_9b0afd792f2b.slice/crio-0c29126eb7b528de2590270cdc34a9431794e9d413e113e75e45a3c6d0475043 WatchSource:0}: Error finding container 0c29126eb7b528de2590270cdc34a9431794e9d413e113e75e45a3c6d0475043: Status 404 returned error can't find the container with id 0c29126eb7b528de2590270cdc34a9431794e9d413e113e75e45a3c6d0475043
Jul 09 14:00:19.694211 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:19.694176    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"0c29126eb7b528de2590270cdc34a9431794e9d413e113e75e45a3c6d0475043"}
Jul 09 14:00:20.704549 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:20.704509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"}
Jul 09 14:00:21.709210 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:21.709175    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337" exitCode=0
Jul 09 14:00:21.709748 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:21.709250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"}
Jul 09 14:00:22.910793 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:22.910753    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 14:00:24.722557 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:24.722512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 14:00:24.725576 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:24.725543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bf2bd8-1417-48bb-93cd-c7e4763d6817-metrics-certs\") pod \"network-metrics-daemon-nzsrt\" (UID: \"60bf2bd8-1417-48bb-93cd-c7e4763d6817\") " pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 14:00:24.772306 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:24.772269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qwpwr\""
Jul 09 14:00:24.780267 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:24.780246    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nzsrt"
Jul 09 14:00:24.936109 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:24.936082    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nzsrt"]
Jul 09 14:00:24.939060 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:00:24.939025    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bf2bd8_1417_48bb_93cd_c7e4763d6817.slice/crio-fef1e288ec57de8d2883cf55f724ffbece4bd72a02660132cca159737c12c709 WatchSource:0}: Error finding container fef1e288ec57de8d2883cf55f724ffbece4bd72a02660132cca159737c12c709: Status 404 returned error can't find the container with id fef1e288ec57de8d2883cf55f724ffbece4bd72a02660132cca159737c12c709
Jul 09 14:00:25.727110 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:25.727063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzsrt" event={"ID":"60bf2bd8-1417-48bb-93cd-c7e4763d6817","Type":"ContainerStarted","Data":"fef1e288ec57de8d2883cf55f724ffbece4bd72a02660132cca159737c12c709"}
Jul 09 14:00:26.739231 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:26.739184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"}
Jul 09 14:00:26.739712 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:26.739284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"}
Jul 09 14:00:27.744693 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:27.744652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzsrt" event={"ID":"60bf2bd8-1417-48bb-93cd-c7e4763d6817","Type":"ContainerStarted","Data":"2cf1774623f668a65a20b8f559fd732c1f0827922c247820bf5b2ea9bb13f7a1"}
Jul 09 14:00:27.745105 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:27.744721    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nzsrt" event={"ID":"60bf2bd8-1417-48bb-93cd-c7e4763d6817","Type":"ContainerStarted","Data":"40a1780fe73d34a7f8f2fa511595a683565e18bf432cde92772f155ba78ff151"}
Jul 09 14:00:27.761107 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:27.761047    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nzsrt" podStartSLOduration=130.847694909 podStartE2EDuration="2m12.761025372s" totalImagesPullingTime="1.913330463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:58:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:27.760531412 +0000 UTC m=+133.301798221" watchObservedRunningTime="2026-07-09 14:00:27.761025372 +0000 UTC m=+133.302292156"
Jul 09 14:00:28.055361 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.055327    2580 scope.go:117] "RemoveContainer" containerID="bc5ea502e8b174c123bc7cb84761e80293b5b712ed22cb15f2bee2206389b16a"
Jul 09 14:00:28.764377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.764340    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:00:28.764828 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.764671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-qskv9" event={"ID":"f8bca19a-836b-4f10-9d32-883deea28646","Type":"ContainerStarted","Data":"9346e4f4755b573f21fd4d6458a0bf8952f1ca636a4044fe3a3525422a5b3d84"}
Jul 09 14:00:28.765462 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.765035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 14:00:28.771224 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.771193    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-qskv9"
Jul 09 14:00:28.785092 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:28.785042    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-qskv9" podStartSLOduration=52.62403415 podStartE2EDuration="54.785022567s" totalImagesPullingTime="2.160988417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 13:59:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:28.783735303 +0000 UTC m=+134.325002088" watchObservedRunningTime="2026-07-09 14:00:28.785022567 +0000 UTC m=+134.326289351"
Jul 09 14:00:29.771672 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:29.771637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"}
Jul 09 14:00:29.772059 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:29.771702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"}
Jul 09 14:00:29.772059 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:29.771732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"}
Jul 09 14:00:29.772059 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:29.771754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerStarted","Data":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"}
Jul 09 14:00:29.805874 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:29.805799    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.7217992989999997 podStartE2EDuration="11.805762915s" totalImagesPullingTime="8.061174189s" totalInitContainerRuntime="22.789427ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:00:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:00:29.803282117 +0000 UTC m=+135.344548905" watchObservedRunningTime="2026-07-09 14:00:29.805762915 +0000 UTC m=+135.347029701"
Jul 09 14:00:34.192642 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:34.192601    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:00:47.931381 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:47.931304    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" containerName="registry" containerID="cri-o://642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a" gracePeriod=30
Jul 09 14:00:48.179822 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.179793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 14:00:48.234536 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234536 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234625    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjqcc\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234679    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.234783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.234727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token\") pod \"8bb59548-7267-4d29-8ba0-617702819a0c\" (UID: \"8bb59548-7267-4d29-8ba0-617702819a0c\") "
Jul 09 14:00:48.235158 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.235122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:00:48.235248 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.235175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:00:48.237528 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.237496    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:00:48.238020 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.237980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc" (OuterVolumeSpecName: "kube-api-access-tjqcc") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "kube-api-access-tjqcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:00:48.238020 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.238001    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:00:48.238141 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.238048    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:00:48.238141 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.238047    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:00:48.243999 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.243963    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "8bb59548-7267-4d29-8ba0-617702819a0c" (UID: "8bb59548-7267-4d29-8ba0-617702819a0c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:00:48.336215 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336179    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-installation-pull-secrets\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336215 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336214    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/8bb59548-7267-4d29-8ba0-617702819a0c-image-registry-private-configuration\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336215 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336225    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-trusted-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336234    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjqcc\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-kube-api-access-tjqcc\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336243    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/8bb59548-7267-4d29-8ba0-617702819a0c-ca-trust-extracted\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336253    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/8bb59548-7267-4d29-8ba0-617702819a0c-registry-certificates\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336260    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-bound-sa-token\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.336453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.336268    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/8bb59548-7267-4d29-8ba0-617702819a0c-registry-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:00:48.840953 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.840915    2580 generic.go:356] "Generic (PLEG): container finished" podID="8bb59548-7267-4d29-8ba0-617702819a0c" containerID="642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a" exitCode=0
Jul 09 14:00:48.841133 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.840963    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d68668684-nj8nx"
Jul 09 14:00:48.841133 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.841000    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" event={"ID":"8bb59548-7267-4d29-8ba0-617702819a0c","Type":"ContainerDied","Data":"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"}
Jul 09 14:00:48.841133 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.841053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d68668684-nj8nx" event={"ID":"8bb59548-7267-4d29-8ba0-617702819a0c","Type":"ContainerDied","Data":"5ce6e2222b1c19484a122d79b48183b029ef0b7c5090eb25958b86f83e0f3950"}
Jul 09 14:00:48.841133 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.841082    2580 scope.go:117] "RemoveContainer" containerID="642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"
Jul 09 14:00:48.850070 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.850053    2580 scope.go:117] "RemoveContainer" containerID="642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"
Jul 09 14:00:48.850357 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:00:48.850336    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a\": container with ID starting with 642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a not found: ID does not exist" containerID="642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"
Jul 09 14:00:48.850396 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.850369    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a"} err="failed to get container status \"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a\": rpc error: code = NotFound desc = could not find container \"642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a\": container with ID starting with 642498cfc3bf61abce7fe2b29e9643b9ebabcf4c1d7523b810709657fa3cb86a not found: ID does not exist"
Jul 09 14:00:48.861619 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.861586    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 14:00:48.865369 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:48.865337    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d68668684-nj8nx"]
Jul 09 14:00:49.060126 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:00:49.060084    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" path="/var/lib/kubelet/pods/8bb59548-7267-4d29-8ba0-617702819a0c/volumes"
Jul 09 14:01:19.191678 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:19.191641    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:19.212423 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:19.212392    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:19.971747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:19.971713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:37.254608 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.254552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:37.255966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.255927    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="prometheus" containerID="cri-o://1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c" gracePeriod=600
Jul 09 14:01:37.256273 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.256250    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1" gracePeriod=600
Jul 09 14:01:37.256443 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.256426    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-web" containerID="cri-o://d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31" gracePeriod=600
Jul 09 14:01:37.256618 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.256598    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy" containerID="cri-o://bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3" gracePeriod=600
Jul 09 14:01:37.256834 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.256813    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="thanos-sidecar" containerID="cri-o://6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e" gracePeriod=600
Jul 09 14:01:37.257010 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.256995    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="config-reloader" containerID="cri-o://9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795" gracePeriod=600
Jul 09 14:01:37.527511 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.527478    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:37.644945 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.644883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.644960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.644988    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645032    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645057    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645079    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645162    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645219    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645356    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645447    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645478    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645513    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.645604 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645604    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gs9rq\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.646206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.646206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.646206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.645874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle\") pod \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\" (UID: \"ee375ef9-3c7c-416a-a804-9b0afd792f2b\") "
Jul 09 14:01:37.646206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.646122    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.646605 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.646553    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.647486 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.647448    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.647621 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.647540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.648210 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648179    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.648291 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.648349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648320    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config" (OuterVolumeSpecName: "config") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.648694 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.648818 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648721    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.648818 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:01:37.648818 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648786    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:01:37.648995 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.648995 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.648964    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:01:37.649669 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.649631    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out" (OuterVolumeSpecName: "config-out") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:01:37.650433 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.650401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.650537 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.650507    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq" (OuterVolumeSpecName: "kube-api-access-gs9rq") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "kube-api-access-gs9rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:01:37.650797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.650779    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.650864 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.650799    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.651225 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.651211    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.660289 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.660262    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config" (OuterVolumeSpecName: "web-config") pod "ee375ef9-3c7c-416a-a804-9b0afd792f2b" (UID: "ee375ef9-3c7c-416a-a804-9b0afd792f2b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:01:37.747492 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747443    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747492 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747487    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747492 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747499    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-metrics-client-certs\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747492 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747510    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-tls-assets\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747519    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747527    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-metrics-client-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747537    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747558    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-db\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747584    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gs9rq\" (UniqueName: \"kubernetes.io/projected/ee375ef9-3c7c-416a-a804-9b0afd792f2b-kube-api-access-gs9rq\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747594    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747602    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747610    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ee375ef9-3c7c-416a-a804-9b0afd792f2b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747618    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config-out\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747626    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-web-config\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747633    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-config\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747641    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-grpc-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747649    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-kube-rbac-proxy\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747658    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-thanos-prometheus-http-client-file\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:37.747773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:37.747667    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ee375ef9-3c7c-416a-a804-9b0afd792f2b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026275    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1" exitCode=0
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026299    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3" exitCode=0
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026306    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31" exitCode=0
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026311    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e" exitCode=0
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026325    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795" exitCode=0
Jul 09 14:01:38.026309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026330    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c" exitCode=0
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026379    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026459    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026500    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026511    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"}
Jul 09 14:01:38.026803 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.026559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"ee375ef9-3c7c-416a-a804-9b0afd792f2b","Type":"ContainerDied","Data":"0c29126eb7b528de2590270cdc34a9431794e9d413e113e75e45a3c6d0475043"}
Jul 09 14:01:38.035442 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.035423    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.043069 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.043047    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.050776 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.050707    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:38.050831 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.050806    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.054252 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.054232    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:38.057991 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.057973    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.065043 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.065013    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.072641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.072611    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.079584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.079549    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.079881 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.079862    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.079941 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.079890    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.079941 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.079909    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.080161 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.080144    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.080219 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080165    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.080219 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080178    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.080406 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.080387    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.080443 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080416    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.080443 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080433    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.080679 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.080651    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.080760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080678    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.080760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080696    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.080935 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.080919    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.080979 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080949    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.080979 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.080963    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.081173 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.081159    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.081218 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081175    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.081218 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081186    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.081394 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:01:38.081378    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.081471 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081401    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.081471 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081421    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.082008 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081773    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.082008 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.081811    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.082186 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082163    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.082247 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082187    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.082487 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082465    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.082487 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082487    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.082764 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082744    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.082842 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.082766    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.083051 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083033    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.083100 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083053    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.083273 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083254    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.083335 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083273    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.083543 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083518    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.083722 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083546    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.083722 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083710    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:38.083838 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083815    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.083893 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.083841    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.084037 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084023    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy"
Jul 09 14:01:38.084089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084044    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy"
Jul 09 14:01:38.084089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084061    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="init-config-reloader"
Jul 09 14:01:38.084089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084073    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="init-config-reloader"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084091    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="prometheus"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084098    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="prometheus"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084106    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="thanos-sidecar"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084111    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="thanos-sidecar"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084116    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-web"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084121    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-web"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084134    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" containerName="registry"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084142    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" containerName="registry"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084065    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084153    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="config-reloader"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084161    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="config-reloader"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084161    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084171    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-thanos"
Jul 09 14:01:38.084179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084179    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-thanos"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084246    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="thanos-sidecar"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084255    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084261    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="prometheus"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084267    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-thanos"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084274    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="kube-rbac-proxy-web"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084280    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bb59548-7267-4d29-8ba0-617702819a0c" containerName="registry"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084285    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" containerName="config-reloader"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084373    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.084679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084389    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.085002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084675    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.085002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084696    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.085002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084913    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.085002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.084931    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.085195 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085177    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.085237 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085197    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.085434 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085405    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.085498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085438    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.085682 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085660    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.085742 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085683    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.085917 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085899    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.085962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.085917    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.086151 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086133    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.086197 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086151    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.086387 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086369    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.086387 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086386    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.086605 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086584    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.086605 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086605    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.086843 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086825    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.086891 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.086843    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.087086 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087070    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.087132 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087086    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.087298 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087284    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.087298 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087297    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.087487 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087470    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.087530 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087487    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.087755 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087736    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.087832 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087757    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.088012 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.087995    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.088056 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088012    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.088233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088214    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.088301 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088235    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.088466 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088446    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.088509 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088466    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.088694 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088676    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.088694 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088693    2580 scope.go:117] "RemoveContainer" containerID="25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"
Jul 09 14:01:38.088866 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088850    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1"} err="failed to get container status \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": rpc error: code = NotFound desc = could not find container \"25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1\": container with ID starting with 25b10873a641b4fb1710c93788d659b04af3c6c01156452bc0d3f82a9a86f3c1 not found: ID does not exist"
Jul 09 14:01:38.088914 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.088865    2580 scope.go:117] "RemoveContainer" containerID="bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"
Jul 09 14:01:38.089095 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089074    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3"} err="failed to get container status \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": rpc error: code = NotFound desc = could not find container \"bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3\": container with ID starting with bc9ec1c03180a1e7fe5ed615748ac93c2ea44482f16087132afd19ebb262f3b3 not found: ID does not exist"
Jul 09 14:01:38.089138 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089095    2580 scope.go:117] "RemoveContainer" containerID="d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"
Jul 09 14:01:38.089299 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089283    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31"} err="failed to get container status \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": rpc error: code = NotFound desc = could not find container \"d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31\": container with ID starting with d040b9bd1b0f9d1481d8b1bb84202e1a413557243736e6534d0ece286619cf31 not found: ID does not exist"
Jul 09 14:01:38.089348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089300    2580 scope.go:117] "RemoveContainer" containerID="6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"
Jul 09 14:01:38.089534 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089518    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e"} err="failed to get container status \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": rpc error: code = NotFound desc = could not find container \"6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e\": container with ID starting with 6d11b9ed4d85a24f8af5f3f637ebd3691661781e1b8586166d85b31718aa2e8e not found: ID does not exist"
Jul 09 14:01:38.089639 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089535    2580 scope.go:117] "RemoveContainer" containerID="9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"
Jul 09 14:01:38.089797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089780    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795"} err="failed to get container status \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": rpc error: code = NotFound desc = could not find container \"9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795\": container with ID starting with 9b6b0c32793bb6820fc4bdc1fb9137fea450be7a208164d564d343d99e9c5795 not found: ID does not exist"
Jul 09 14:01:38.089840 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089797    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.089895 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.089797    2580 scope.go:117] "RemoveContainer" containerID="1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"
Jul 09 14:01:38.090179 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.090152    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c"} err="failed to get container status \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": rpc error: code = NotFound desc = could not find container \"1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c\": container with ID starting with 1aa4a8a9d0ce386eebdc34b2b74d7dd90d2803f158255c136b7b2842ade3ea1c not found: ID does not exist"
Jul 09 14:01:38.090281 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.090181    2580 scope.go:117] "RemoveContainer" containerID="1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"
Jul 09 14:01:38.090489 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.090464    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337"} err="failed to get container status \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": rpc error: code = NotFound desc = could not find container \"1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337\": container with ID starting with 1be4853c332005234ab83a12af27cb500613b8f6eceec71ed39512de20f66337 not found: ID does not exist"
Jul 09 14:01:38.092432 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 09 14:01:38.092528 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 09 14:01:38.092528 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092411    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 09 14:01:38.092662 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092534    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 09 14:01:38.092662 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092587    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 09 14:01:38.092764 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092702    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 09 14:01:38.092951 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092907    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 09 14:01:38.093034 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4id6ojmm6pv95\""
Jul 09 14:01:38.093089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.093039    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 09 14:01:38.093089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.093052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 09 14:01:38.093174 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.092942    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 09 14:01:38.093373 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.093358    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 09 14:01:38.093450 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.093420    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-kpgh9\""
Jul 09 14:01:38.093505 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.093480    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 09 14:01:38.095811 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.095791    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 09 14:01:38.103716 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.103690    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 09 14:01:38.105211 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.105146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 09 14:01:38.106036 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.106014    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:38.251021 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.250981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251021 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251021    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config-out\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251228    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251272 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-web-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251285    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4zwq\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-kube-api-access-h4zwq\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251594    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251933 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.251933 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.251664    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.352997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.352954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config-out\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.352997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.352995    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-web-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4zwq\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-kube-api-access-h4zwq\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353312    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353495    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.354714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.355510 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.353620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.355510 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.354164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.355510 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.354165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.356122 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.356087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config-out\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.356400 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.356372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.356485 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.356435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-web-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.356485 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.356474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.357776 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.357110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.357776 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.357530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.357776 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.357687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358042 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358018    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358112 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358195 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358762 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358710    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358887 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/51f5e2d2-2fb6-41fc-ab4b-06f940617333-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.358887 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.358834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.359102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.359081    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.359191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.359171    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.360663 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.360644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-config\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.361099 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.361077    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/51f5e2d2-2fb6-41fc-ab4b-06f940617333-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.361748 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.361727    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4zwq\" (UniqueName: \"kubernetes.io/projected/51f5e2d2-2fb6-41fc-ab4b-06f940617333-kube-api-access-h4zwq\") pod \"prometheus-k8s-0\" (UID: \"51f5e2d2-2fb6-41fc-ab4b-06f940617333\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.404762 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.404709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:01:38.555321 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:38.555286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 09 14:01:38.558967 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:01:38.558938    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51f5e2d2_2fb6_41fc_ab4b_06f940617333.slice/crio-84509fbe840a2705cf2630538ab60d4b415593f948639384bb158b218a4fd22e WatchSource:0}: Error finding container 84509fbe840a2705cf2630538ab60d4b415593f948639384bb158b218a4fd22e: Status 404 returned error can't find the container with id 84509fbe840a2705cf2630538ab60d4b415593f948639384bb158b218a4fd22e
Jul 09 14:01:39.031935 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:39.031831    2580 generic.go:356] "Generic (PLEG): container finished" podID="51f5e2d2-2fb6-41fc-ab4b-06f940617333" containerID="44f4002a5d33f5d500c8af819f2415d537eb8e5f7aa7ea6f49e7c17c769a12e4" exitCode=0
Jul 09 14:01:39.032089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:39.031927    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerDied","Data":"44f4002a5d33f5d500c8af819f2415d537eb8e5f7aa7ea6f49e7c17c769a12e4"}
Jul 09 14:01:39.032089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:39.032003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"84509fbe840a2705cf2630538ab60d4b415593f948639384bb158b218a4fd22e"}
Jul 09 14:01:39.060897 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:39.060866    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee375ef9-3c7c-416a-a804-9b0afd792f2b" path="/var/lib/kubelet/pods/ee375ef9-3c7c-416a-a804-9b0afd792f2b/volumes"
Jul 09 14:01:40.040478 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040435    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"bd218681c58063249a8dd9e28e0d108a40d080b7cf8c9f5d4fd260ecf5d1af48"}
Jul 09 14:01:40.040863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"8591c5477f0698d210d22666436e75a1db37259839c2606b9a4252a0742ea90b"}
Jul 09 14:01:40.040863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"296fc492e1a3418301bbff781a1b2fdb7d318d569c2f041e6209581de2f49fd4"}
Jul 09 14:01:40.040863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040616    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"7ec1b63cd755e4cb0a90f198ba5cb4b8cd4e7920b63eed8bc7f7f87e8c338319"}
Jul 09 14:01:40.040863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"82d7d7343baa399df10841e2126695a8b151285e65581441a885c97b37d77a74"}
Jul 09 14:01:40.040863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.040664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"51f5e2d2-2fb6-41fc-ab4b-06f940617333","Type":"ContainerStarted","Data":"e06bf6221afde7daa7ab9119d5f002a1f6f902e0541cdb3c77550b6ac18c192d"}
Jul 09 14:01:40.068269 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:40.068208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.048919294 podStartE2EDuration="2.068188314s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.26902ms" isStatefulPod=false podCreationTimestamp="2026-07-09 14:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:01:40.066985367 +0000 UTC m=+205.608252194" watchObservedRunningTime="2026-07-09 14:01:40.068188314 +0000 UTC m=+205.609455097"
Jul 09 14:01:43.405258 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:01:43.405214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:38.405903 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:02:38.405807    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:38.422175 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:02:38.422142    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:02:39.278746 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:02:39.278706    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 09 14:03:05.498480 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.498441    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7lxpl"]
Jul 09 14:03:05.501802 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.501779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.503958 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.503940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qvrl5\""
Jul 09 14:03:05.504067 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.504047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 14:03:05.504756 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.504741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 14:03:05.511268 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.511248    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7lxpl"]
Jul 09 14:03:05.535273 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.535242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-bound-sa-token\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.535422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.535303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8hvz\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-kube-api-access-b8hvz\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.635876 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.635839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8hvz\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-kube-api-access-b8hvz\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.636039 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.635898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-bound-sa-token\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.643852 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.643821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8hvz\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-kube-api-access-b8hvz\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.644465 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.644446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e7531f59-6319-4c1d-b633-d24b92f07314-bound-sa-token\") pod \"cert-manager-55b4d954d5-7lxpl\" (UID: \"e7531f59-6319-4c1d-b633-d24b92f07314\") " pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.822374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.822333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-7lxpl"
Jul 09 14:03:05.953360 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:05.953322    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-7lxpl"]
Jul 09 14:03:05.956653 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:03:05.956617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7531f59_6319_4c1d_b633_d24b92f07314.slice/crio-52e56bf859ce76a90a15b9b717fc154da881ffdbd019554c1c72450cce2233ef WatchSource:0}: Error finding container 52e56bf859ce76a90a15b9b717fc154da881ffdbd019554c1c72450cce2233ef: Status 404 returned error can't find the container with id 52e56bf859ce76a90a15b9b717fc154da881ffdbd019554c1c72450cce2233ef
Jul 09 14:03:06.357388 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:06.357350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7lxpl" event={"ID":"e7531f59-6319-4c1d-b633-d24b92f07314","Type":"ContainerStarted","Data":"52e56bf859ce76a90a15b9b717fc154da881ffdbd019554c1c72450cce2233ef"}
Jul 09 14:03:07.418277 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.418205    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"]
Jul 09 14:03:07.428093 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.427541    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.428274 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.428112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"]
Jul 09 14:03:07.429985 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.429957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bjwd8\""
Jul 09 14:03:07.431264 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.431048    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 14:03:07.431264 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.431047    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 14:03:07.449358 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.449330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.449532 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.449434    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xb5j\" (UniqueName: \"kubernetes.io/projected/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-kube-api-access-7xb5j\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.550247 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.550204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.550454 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.550322    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xb5j\" (UniqueName: \"kubernetes.io/projected/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-kube-api-access-7xb5j\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.550631 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.550609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.560127 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.560091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xb5j\" (UniqueName: \"kubernetes.io/projected/6393ec93-7ac6-4c28-a21b-08cb58c3f84a-kube-api-access-7xb5j\") pod \"openshift-lws-operator-fd8ccff4c-c87q4\" (UID: \"6393ec93-7ac6-4c28-a21b-08cb58c3f84a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:07.740694 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:07.740602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"
Jul 09 14:03:08.931254 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:08.931226    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4"]
Jul 09 14:03:08.933742 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:03:08.933709    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6393ec93_7ac6_4c28_a21b_08cb58c3f84a.slice/crio-f1bc440969a649b5a74bb0ae6adca155159ccdfccd0d49083d477277950f702c WatchSource:0}: Error finding container f1bc440969a649b5a74bb0ae6adca155159ccdfccd0d49083d477277950f702c: Status 404 returned error can't find the container with id f1bc440969a649b5a74bb0ae6adca155159ccdfccd0d49083d477277950f702c
Jul 09 14:03:09.371625 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:09.371559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-7lxpl" event={"ID":"e7531f59-6319-4c1d-b633-d24b92f07314","Type":"ContainerStarted","Data":"b38cfc9aae00ae63b11933a5a1b2cd4ad7cb48d470dd7268ea2612db21a5acea"}
Jul 09 14:03:09.372984 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:09.372958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4" event={"ID":"6393ec93-7ac6-4c28-a21b-08cb58c3f84a","Type":"ContainerStarted","Data":"f1bc440969a649b5a74bb0ae6adca155159ccdfccd0d49083d477277950f702c"}
Jul 09 14:03:09.387987 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:09.387937    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-7lxpl" podStartSLOduration=1.478234836 podStartE2EDuration="4.387923931s" totalImagesPullingTime="2.909689095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:09.387083592 +0000 UTC m=+294.928350378" watchObservedRunningTime="2026-07-09 14:03:09.387923931 +0000 UTC m=+294.929190713"
Jul 09 14:03:14.397746 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.397704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4" event={"ID":"6393ec93-7ac6-4c28-a21b-08cb58c3f84a","Type":"ContainerStarted","Data":"eb05af776dab527ce33101f9a7841a0bd6e6fbd075632ad297caf20f6b10df03"}
Jul 09 14:03:14.413719 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.413672    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-c87q4" podStartSLOduration=2.436789068 podStartE2EDuration="7.413655311s" totalImagesPullingTime="4.976866243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:14.413192667 +0000 UTC m=+299.954459450" watchObservedRunningTime="2026-07-09 14:03:14.413655311 +0000 UTC m=+299.954922094"
Jul 09 14:03:14.875680 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.875655    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:03:14.886927 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.886877    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:03:14.891991 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.891968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:03:14.892741 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.892721    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 14:03:14.898453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:14.898432    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:03:29.235679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.235644    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"]
Jul 09 14:03:29.315769 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.315723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"]
Jul 09 14:03:29.315875 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.315847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.318952 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.318921    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 09 14:03:29.319875 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.319850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cvjsw\""
Jul 09 14:03:29.319971 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.319850    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 09 14:03:29.319971 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.319892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 09 14:03:29.431541 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.431502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-metrics-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.431763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.431563    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ada3c7-a764-4e53-8b27-bcc3c75a307b-manager-config\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.431763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.431628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8jnm\" (UniqueName: \"kubernetes.io/projected/37ada3c7-a764-4e53-8b27-bcc3c75a307b-kube-api-access-r8jnm\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.431763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.431661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.532256 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.532167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.532256 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.532219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-metrics-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.532522 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.532279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ada3c7-a764-4e53-8b27-bcc3c75a307b-manager-config\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.532522 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.532296    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8jnm\" (UniqueName: \"kubernetes.io/projected/37ada3c7-a764-4e53-8b27-bcc3c75a307b-kube-api-access-r8jnm\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.533095 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.533068    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/37ada3c7-a764-4e53-8b27-bcc3c75a307b-manager-config\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.535609 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.535582    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.535723 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.535667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/37ada3c7-a764-4e53-8b27-bcc3c75a307b-metrics-cert\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.541637 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.541609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8jnm\" (UniqueName: \"kubernetes.io/projected/37ada3c7-a764-4e53-8b27-bcc3c75a307b-kube-api-access-r8jnm\") pod \"lws-controller-manager-86df669b44-hdnvf\" (UID: \"37ada3c7-a764-4e53-8b27-bcc3c75a307b\") " pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.627335 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.627296    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:29.768980 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.768950    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"]
Jul 09 14:03:29.771510 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:03:29.771468    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ada3c7_a764_4e53_8b27_bcc3c75a307b.slice/crio-59fff2f78394c17cfbdb6a395f1f8f7037ac641b50b2e35e2e21b0300e40cb4c WatchSource:0}: Error finding container 59fff2f78394c17cfbdb6a395f1f8f7037ac641b50b2e35e2e21b0300e40cb4c: Status 404 returned error can't find the container with id 59fff2f78394c17cfbdb6a395f1f8f7037ac641b50b2e35e2e21b0300e40cb4c
Jul 09 14:03:29.773530 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:29.773508    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:03:30.463893 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:30.463852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf" event={"ID":"37ada3c7-a764-4e53-8b27-bcc3c75a307b","Type":"ContainerStarted","Data":"59fff2f78394c17cfbdb6a395f1f8f7037ac641b50b2e35e2e21b0300e40cb4c"}
Jul 09 14:03:31.474330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:31.469925    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf" event={"ID":"37ada3c7-a764-4e53-8b27-bcc3c75a307b","Type":"ContainerStarted","Data":"02832790571b41fa739c5c886c0ad7bdebef9d6b7d32883511f00b50183b0715"}
Jul 09 14:03:31.474330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:31.470703    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:31.511450 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:31.511393    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf" podStartSLOduration=0.935071471 podStartE2EDuration="2.511374608s" totalImagesPullingTime="1.576303137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:31.510107301 +0000 UTC m=+317.051374083" watchObservedRunningTime="2026-07-09 14:03:31.511374608 +0000 UTC m=+317.052641391"
Jul 09 14:03:34.236662 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.236633    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"]
Jul 09 14:03:34.239916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.239901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.243249 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.243230    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 14:03:34.243372 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.243302    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-shds6\""
Jul 09 14:03:34.243430 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.243372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 14:03:34.243781 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.243762    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 14:03:34.245354 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.245325    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 14:03:34.267047 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.267020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"]
Jul 09 14:03:34.376174 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.376143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-tmp\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.376174 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.376176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-manifests\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.376428 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.376207    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.376428 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.376348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-webhook-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.376428 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.376382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm4t8\" (UniqueName: \"kubernetes.io/projected/2bc0e30d-568f-4b2f-831e-c9a79af58740-kube-api-access-bm4t8\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.477708 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.477679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.477868 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.477741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-webhook-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.477868 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.477761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm4t8\" (UniqueName: \"kubernetes.io/projected/2bc0e30d-568f-4b2f-831e-c9a79af58740-kube-api-access-bm4t8\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.477868 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.477785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-tmp\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.477868 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.477801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-manifests\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.478288 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.478263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-manifests\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.480502 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.480468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-apiservice-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.480665 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.480646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2bc0e30d-568f-4b2f-831e-c9a79af58740-tmp\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.480763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.480746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2bc0e30d-568f-4b2f-831e-c9a79af58740-webhook-cert\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.485407 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.485386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm4t8\" (UniqueName: \"kubernetes.io/projected/2bc0e30d-568f-4b2f-831e-c9a79af58740-kube-api-access-bm4t8\") pod \"opendatahub-operator-controller-manager-78fdd99dcd-4trrn\" (UID: \"2bc0e30d-568f-4b2f-831e-c9a79af58740\") " pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.549834 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.549754    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:34.683808 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:34.683773    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"]
Jul 09 14:03:34.686882 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:03:34.686852    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bc0e30d_568f_4b2f_831e_c9a79af58740.slice/crio-5364eddeef91bfe0d9db2ac88f6b848ab6a53733d143cee8d807ea7d25b2055c WatchSource:0}: Error finding container 5364eddeef91bfe0d9db2ac88f6b848ab6a53733d143cee8d807ea7d25b2055c: Status 404 returned error can't find the container with id 5364eddeef91bfe0d9db2ac88f6b848ab6a53733d143cee8d807ea7d25b2055c
Jul 09 14:03:35.486229 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:35.486193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn" event={"ID":"2bc0e30d-568f-4b2f-831e-c9a79af58740","Type":"ContainerStarted","Data":"5364eddeef91bfe0d9db2ac88f6b848ab6a53733d143cee8d807ea7d25b2055c"}
Jul 09 14:03:40.510951 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:40.510917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn" event={"ID":"2bc0e30d-568f-4b2f-831e-c9a79af58740","Type":"ContainerStarted","Data":"8b717a7a08a12bcabe7e53aa4f1f3ac23881e29e500d82c1715879e178292ae4"}
Jul 09 14:03:41.517133 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:41.517096    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bc0e30d-568f-4b2f-831e-c9a79af58740" containerID="8b717a7a08a12bcabe7e53aa4f1f3ac23881e29e500d82c1715879e178292ae4" exitCode=0
Jul 09 14:03:41.517590 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:41.517184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn" event={"ID":"2bc0e30d-568f-4b2f-831e-c9a79af58740","Type":"ContainerDied","Data":"8b717a7a08a12bcabe7e53aa4f1f3ac23881e29e500d82c1715879e178292ae4"}
Jul 09 14:03:42.476977 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:42.476947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-86df669b44-hdnvf"
Jul 09 14:03:44.532382 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:44.532346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn" event={"ID":"2bc0e30d-568f-4b2f-831e-c9a79af58740","Type":"ContainerStarted","Data":"fe49c99cc18b50282f13e17ace49aea559f89140082bf3a9b40d496886e2a3c5"}
Jul 09 14:03:44.532884 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:44.532453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:44.556981 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:44.556921    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn" podStartSLOduration=1.8071751059999999 podStartE2EDuration="10.556903586s" totalImagesPullingTime="7.711851881s" totalInitContainerRuntime="1.037876599s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:03:44.553296486 +0000 UTC m=+330.094563326" watchObservedRunningTime="2026-07-09 14:03:44.556903586 +0000 UTC m=+330.098170371"
Jul 09 14:03:55.539921 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:55.539885    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-78fdd99dcd-4trrn"
Jul 09 14:03:57.888665 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.888619    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"]
Jul 09 14:03:57.897964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.897928    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:57.901282 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.901244    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 14:03:57.902544 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.902487    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 14:03:57.902852 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.902632    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 14:03:57.903018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.902703    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 14:03:57.903194 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.902734    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-b9nrc\""
Jul 09 14:03:57.904724 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.904687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"]
Jul 09 14:03:57.979508 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.979475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18ed8de9-6c28-4975-8b96-7858cec7279f-tmp\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:57.979508 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.979513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g86l6\" (UniqueName: \"kubernetes.io/projected/18ed8de9-6c28-4975-8b96-7858cec7279f-kube-api-access-g86l6\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:57.979760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:57.979640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ed8de9-6c28-4975-8b96-7858cec7279f-tls-certs\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.080250 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.080215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ed8de9-6c28-4975-8b96-7858cec7279f-tls-certs\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.080461 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.080292    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18ed8de9-6c28-4975-8b96-7858cec7279f-tmp\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.080461 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.080310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g86l6\" (UniqueName: \"kubernetes.io/projected/18ed8de9-6c28-4975-8b96-7858cec7279f-kube-api-access-g86l6\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.082834 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.082808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18ed8de9-6c28-4975-8b96-7858cec7279f-tmp\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.083085 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.083062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18ed8de9-6c28-4975-8b96-7858cec7279f-tls-certs\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.096427 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.096385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g86l6\" (UniqueName: \"kubernetes.io/projected/18ed8de9-6c28-4975-8b96-7858cec7279f-kube-api-access-g86l6\") pod \"kube-auth-proxy-78985cb57d-4hxxk\" (UID: \"18ed8de9-6c28-4975-8b96-7858cec7279f\") " pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.211130 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.211035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"
Jul 09 14:03:58.361045 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.361005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk"]
Jul 09 14:03:58.364070 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:03:58.364035    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18ed8de9_6c28_4975_8b96_7858cec7279f.slice/crio-d51d24dbc9a0d816f3423977835fddcbdfb34e48da8c2f0373412b2f5285acf3 WatchSource:0}: Error finding container d51d24dbc9a0d816f3423977835fddcbdfb34e48da8c2f0373412b2f5285acf3: Status 404 returned error can't find the container with id d51d24dbc9a0d816f3423977835fddcbdfb34e48da8c2f0373412b2f5285acf3
Jul 09 14:03:58.609196 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:03:58.609147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk" event={"ID":"18ed8de9-6c28-4975-8b96-7858cec7279f","Type":"ContainerStarted","Data":"d51d24dbc9a0d816f3423977835fddcbdfb34e48da8c2f0373412b2f5285acf3"}
Jul 09 14:04:02.632057 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:04:02.632020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk" event={"ID":"18ed8de9-6c28-4975-8b96-7858cec7279f","Type":"ContainerStarted","Data":"397c400a77261ce9a44f3d2d45cd5ee917cc45ff97fa331c73ef7ba084156cf7"}
Jul 09 14:04:02.650470 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:04:02.650412    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-78985cb57d-4hxxk" podStartSLOduration=2.103535345 podStartE2EDuration="5.650397333s" totalImagesPullingTime="3.546861988s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:03:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:04:02.648532223 +0000 UTC m=+348.189799021" watchObservedRunningTime="2026-07-09 14:04:02.650397333 +0000 UTC m=+348.191664116"
Jul 09 14:06:37.918698 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.918652    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:37.920870 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.920849    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:37.923454 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.923424    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 14:06:37.923454 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.923430    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 14:06:37.924222 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.924202    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 14:06:37.924327 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.924204    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ddbx2\""
Jul 09 14:06:37.930686 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:37.930660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:38.017146 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.017113    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:38.055027 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.054993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.055027 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.055039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm8fn\" (UniqueName: \"kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.156056 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.155931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.156345 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.156113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm8fn\" (UniqueName: \"kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.156640 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.156620    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.165302 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.165263    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm8fn\" (UniqueName: \"kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn\") pod \"limitador-limitador-64c94fd64f-4kg4h\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.232174 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.232074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:38.372903 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.372870    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:38.376355 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:06:38.376328    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26bb057e_44c4_47f0_915b_3d11aa223416.slice/crio-53082100adec73125527fb3f77ca84333ff9f2333f33ec2a0e5442ce976e7014 WatchSource:0}: Error finding container 53082100adec73125527fb3f77ca84333ff9f2333f33ec2a0e5442ce976e7014: Status 404 returned error can't find the container with id 53082100adec73125527fb3f77ca84333ff9f2333f33ec2a0e5442ce976e7014
Jul 09 14:06:38.837957 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.837924    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:38.840908 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.840885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:38.843366 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.843344    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c7ptb\""
Jul 09 14:06:38.847112 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.847088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:38.963965 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:38.963928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns46n\" (UniqueName: \"kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n\") pod \"authorino-58cd955675-kslpl\" (UID: \"6641a410-1b23-422b-8090-285eae419492\") " pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:39.037094 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.037060    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:06:39.039157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.039140    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:06:39.046731 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.046708    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:06:39.064520 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.064480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns46n\" (UniqueName: \"kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n\") pod \"authorino-58cd955675-kslpl\" (UID: \"6641a410-1b23-422b-8090-285eae419492\") " pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:39.072371 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.072343    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns46n\" (UniqueName: \"kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n\") pod \"authorino-58cd955675-kslpl\" (UID: \"6641a410-1b23-422b-8090-285eae419492\") " pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:39.152410 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.152238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:39.166124 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.165502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqr6x\" (UniqueName: \"kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x\") pod \"authorino-75d556bd77-hxq5p\" (UID: \"0dc079c0-54ba-433d-a4eb-4f9be563e965\") " pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:06:39.267555 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.267436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hqr6x\" (UniqueName: \"kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x\") pod \"authorino-75d556bd77-hxq5p\" (UID: \"0dc079c0-54ba-433d-a4eb-4f9be563e965\") " pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:06:39.286275 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.285431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqr6x\" (UniqueName: \"kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x\") pod \"authorino-75d556bd77-hxq5p\" (UID: \"0dc079c0-54ba-433d-a4eb-4f9be563e965\") " pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:06:39.303708 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.303658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" event={"ID":"26bb057e-44c4-47f0-915b-3d11aa223416","Type":"ContainerStarted","Data":"53082100adec73125527fb3f77ca84333ff9f2333f33ec2a0e5442ce976e7014"}
Jul 09 14:06:39.336159 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.336118    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:39.341779 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:06:39.341737    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6641a410_1b23_422b_8090_285eae419492.slice/crio-6309b3ad83aad9bf76d008a118c967544fdef00b47f20a9b4e71fa7340a03a0a WatchSource:0}: Error finding container 6309b3ad83aad9bf76d008a118c967544fdef00b47f20a9b4e71fa7340a03a0a: Status 404 returned error can't find the container with id 6309b3ad83aad9bf76d008a118c967544fdef00b47f20a9b4e71fa7340a03a0a
Jul 09 14:06:39.351419 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.351073    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:06:39.556149 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:39.556115    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:06:39.559441 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:06:39.559404    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0dc079c0_54ba_433d_a4eb_4f9be563e965.slice/crio-68ff1194fa7a0307bf86a97c51132b493202530b04021eaaa80b2e6f5d0e3ac1 WatchSource:0}: Error finding container 68ff1194fa7a0307bf86a97c51132b493202530b04021eaaa80b2e6f5d0e3ac1: Status 404 returned error can't find the container with id 68ff1194fa7a0307bf86a97c51132b493202530b04021eaaa80b2e6f5d0e3ac1
Jul 09 14:06:40.317766 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:40.317662    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hxq5p" event={"ID":"0dc079c0-54ba-433d-a4eb-4f9be563e965","Type":"ContainerStarted","Data":"68ff1194fa7a0307bf86a97c51132b493202530b04021eaaa80b2e6f5d0e3ac1"}
Jul 09 14:06:40.321251 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:40.321213    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kslpl" event={"ID":"6641a410-1b23-422b-8090-285eae419492","Type":"ContainerStarted","Data":"6309b3ad83aad9bf76d008a118c967544fdef00b47f20a9b4e71fa7340a03a0a"}
Jul 09 14:06:44.346705 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.346665    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" event={"ID":"26bb057e-44c4-47f0-915b-3d11aa223416","Type":"ContainerStarted","Data":"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"}
Jul 09 14:06:44.347330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.346800    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:44.348498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.348468    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kslpl" event={"ID":"6641a410-1b23-422b-8090-285eae419492","Type":"ContainerStarted","Data":"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"}
Jul 09 14:06:44.350186 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.350159    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hxq5p" event={"ID":"0dc079c0-54ba-433d-a4eb-4f9be563e965","Type":"ContainerStarted","Data":"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"}
Jul 09 14:06:44.363224 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.363164    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" podStartSLOduration=2.087610557 podStartE2EDuration="7.363146826s" totalImagesPullingTime="5.275536269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:44.362616692 +0000 UTC m=+509.903883497" watchObservedRunningTime="2026-07-09 14:06:44.363146826 +0000 UTC m=+509.904413608"
Jul 09 14:06:44.377386 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.377269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kslpl" podStartSLOduration=2.078923042 podStartE2EDuration="6.377250478s" totalImagesPullingTime="4.298327436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:44.375345174 +0000 UTC m=+509.916611959" watchObservedRunningTime="2026-07-09 14:06:44.377250478 +0000 UTC m=+509.918517261"
Jul 09 14:06:44.387785 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.387720    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-hxq5p" podStartSLOduration=1.335343371 podStartE2EDuration="5.38769709s" totalImagesPullingTime="4.052353719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:06:44.386828443 +0000 UTC m=+509.928095227" watchObservedRunningTime="2026-07-09 14:06:44.38769709 +0000 UTC m=+509.928963874"
Jul 09 14:06:44.413225 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:44.413187    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:46.358487 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:46.358434    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kslpl" podUID="6641a410-1b23-422b-8090-285eae419492" containerName="authorino" containerID="cri-o://735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538" gracePeriod=30
Jul 09 14:06:46.614406 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:46.614331    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:46.629896 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:46.629867    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns46n\" (UniqueName: \"kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n\") pod \"6641a410-1b23-422b-8090-285eae419492\" (UID: \"6641a410-1b23-422b-8090-285eae419492\") "
Jul 09 14:06:46.632784 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:46.632744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n" (OuterVolumeSpecName: "kube-api-access-ns46n") pod "6641a410-1b23-422b-8090-285eae419492" (UID: "6641a410-1b23-422b-8090-285eae419492"). InnerVolumeSpecName "kube-api-access-ns46n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:06:46.731316 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:46.731256    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns46n\" (UniqueName: \"kubernetes.io/projected/6641a410-1b23-422b-8090-285eae419492-kube-api-access-ns46n\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:06:47.365529 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.365425    2580 generic.go:356] "Generic (PLEG): container finished" podID="6641a410-1b23-422b-8090-285eae419492" containerID="735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538" exitCode=0
Jul 09 14:06:47.365529 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.365471    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kslpl"
Jul 09 14:06:47.365529 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.365506    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kslpl" event={"ID":"6641a410-1b23-422b-8090-285eae419492","Type":"ContainerDied","Data":"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"}
Jul 09 14:06:47.366049 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.365555    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kslpl" event={"ID":"6641a410-1b23-422b-8090-285eae419492","Type":"ContainerDied","Data":"6309b3ad83aad9bf76d008a118c967544fdef00b47f20a9b4e71fa7340a03a0a"}
Jul 09 14:06:47.366049 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.365603    2580 scope.go:117] "RemoveContainer" containerID="735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"
Jul 09 14:06:47.374911 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.374888    2580 scope.go:117] "RemoveContainer" containerID="735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"
Jul 09 14:06:47.375239 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:06:47.375218    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538\": container with ID starting with 735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538 not found: ID does not exist" containerID="735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"
Jul 09 14:06:47.375305 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.375253    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538"} err="failed to get container status \"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538\": rpc error: code = NotFound desc = could not find container \"735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538\": container with ID starting with 735209284f1f187755dd2b841e6d5d709e2a3c0fe39d0e0235abaa44b36c4538 not found: ID does not exist"
Jul 09 14:06:47.380465 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.380437    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:47.384498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:47.384473    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kslpl"]
Jul 09 14:06:49.060279 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:49.060238    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6641a410-1b23-422b-8090-285eae419492" path="/var/lib/kubelet/pods/6641a410-1b23-422b-8090-285eae419492/volumes"
Jul 09 14:06:52.536508 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:52.536457    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:52.536985 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:52.536754    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" podUID="26bb057e-44c4-47f0-915b-3d11aa223416" containerName="limitador" containerID="cri-o://a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71" gracePeriod=30
Jul 09 14:06:52.538016 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:52.537992    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:53.105494 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.105426    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:53.190885 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.190844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm8fn\" (UniqueName: \"kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn\") pod \"26bb057e-44c4-47f0-915b-3d11aa223416\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") "
Jul 09 14:06:53.191082 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.190936    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file\") pod \"26bb057e-44c4-47f0-915b-3d11aa223416\" (UID: \"26bb057e-44c4-47f0-915b-3d11aa223416\") "
Jul 09 14:06:53.191321 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.191297    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file" (OuterVolumeSpecName: "config-file") pod "26bb057e-44c4-47f0-915b-3d11aa223416" (UID: "26bb057e-44c4-47f0-915b-3d11aa223416"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:06:53.193249 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.193219    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn" (OuterVolumeSpecName: "kube-api-access-wm8fn") pod "26bb057e-44c4-47f0-915b-3d11aa223416" (UID: "26bb057e-44c4-47f0-915b-3d11aa223416"). InnerVolumeSpecName "kube-api-access-wm8fn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:06:53.291984 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.291932    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm8fn\" (UniqueName: \"kubernetes.io/projected/26bb057e-44c4-47f0-915b-3d11aa223416-kube-api-access-wm8fn\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:06:53.291984 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.291972    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/26bb057e-44c4-47f0-915b-3d11aa223416-config-file\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:06:53.394885 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.394782    2580 generic.go:356] "Generic (PLEG): container finished" podID="26bb057e-44c4-47f0-915b-3d11aa223416" containerID="a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71" exitCode=0
Jul 09 14:06:53.394885 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.394831    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"
Jul 09 14:06:53.394885 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.394863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" event={"ID":"26bb057e-44c4-47f0-915b-3d11aa223416","Type":"ContainerDied","Data":"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"}
Jul 09 14:06:53.395142 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.394918    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-4kg4h" event={"ID":"26bb057e-44c4-47f0-915b-3d11aa223416","Type":"ContainerDied","Data":"53082100adec73125527fb3f77ca84333ff9f2333f33ec2a0e5442ce976e7014"}
Jul 09 14:06:53.395142 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.394945    2580 scope.go:117] "RemoveContainer" containerID="a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"
Jul 09 14:06:53.404088 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.404070    2580 scope.go:117] "RemoveContainer" containerID="a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"
Jul 09 14:06:53.404389 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:06:53.404369    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71\": container with ID starting with a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71 not found: ID does not exist" containerID="a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"
Jul 09 14:06:53.404475 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.404397    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71"} err="failed to get container status \"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71\": rpc error: code = NotFound desc = could not find container \"a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71\": container with ID starting with a40902212bbba2a73061ece18128b149dfc85f4c6ec0dbb6feaa9be4a1af7c71 not found: ID does not exist"
Jul 09 14:06:53.415523 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.415493    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:53.419357 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:53.419326    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-4kg4h"]
Jul 09 14:06:54.435099 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.435057    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"]
Jul 09 14:06:54.435830 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.435807    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6641a410-1b23-422b-8090-285eae419492" containerName="authorino"
Jul 09 14:06:54.435913 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.435833    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6641a410-1b23-422b-8090-285eae419492" containerName="authorino"
Jul 09 14:06:54.435913 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.435846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26bb057e-44c4-47f0-915b-3d11aa223416" containerName="limitador"
Jul 09 14:06:54.435913 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.435855    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="26bb057e-44c4-47f0-915b-3d11aa223416" containerName="limitador"
Jul 09 14:06:54.436060 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.436046    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6641a410-1b23-422b-8090-285eae419492" containerName="authorino"
Jul 09 14:06:54.436107 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.436070    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="26bb057e-44c4-47f0-915b-3d11aa223416" containerName="limitador"
Jul 09 14:06:54.441735 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.441704    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.444626 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.444497    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-mppkm\""
Jul 09 14:06:54.444626 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.444497    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 14:06:54.444626 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.444608    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 14:06:54.445981 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.445813    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 14:06:54.447699 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.447672    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"]
Jul 09 14:06:54.501016 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.500976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38858016-310b-44cb-b5be-957a055a2e35-data\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.501016 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.501016    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7mc6\" (UniqueName: \"kubernetes.io/projected/38858016-310b-44cb-b5be-957a055a2e35-kube-api-access-k7mc6\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.601966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.601926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38858016-310b-44cb-b5be-957a055a2e35-data\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.601966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.601971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7mc6\" (UniqueName: \"kubernetes.io/projected/38858016-310b-44cb-b5be-957a055a2e35-kube-api-access-k7mc6\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.602352 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.602329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/38858016-310b-44cb-b5be-957a055a2e35-data\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.610307 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.610271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7mc6\" (UniqueName: \"kubernetes.io/projected/38858016-310b-44cb-b5be-957a055a2e35-kube-api-access-k7mc6\") pod \"postgres-69fb4594cf-9dbcv\" (UID: \"38858016-310b-44cb-b5be-957a055a2e35\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.755395 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.755287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:06:54.898065 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:54.898033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"]
Jul 09 14:06:54.900809 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:06:54.900781    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38858016_310b_44cb_b5be_957a055a2e35.slice/crio-2e3623cc4b7445e28435a93c9765f405d2c56bb98912e62ccf615ccbefb30230 WatchSource:0}: Error finding container 2e3623cc4b7445e28435a93c9765f405d2c56bb98912e62ccf615ccbefb30230: Status 404 returned error can't find the container with id 2e3623cc4b7445e28435a93c9765f405d2c56bb98912e62ccf615ccbefb30230
Jul 09 14:06:55.060325 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:55.060285    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26bb057e-44c4-47f0-915b-3d11aa223416" path="/var/lib/kubelet/pods/26bb057e-44c4-47f0-915b-3d11aa223416/volumes"
Jul 09 14:06:55.407189 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:06:55.407093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv" event={"ID":"38858016-310b-44cb-b5be-957a055a2e35","Type":"ContainerStarted","Data":"2e3623cc4b7445e28435a93c9765f405d2c56bb98912e62ccf615ccbefb30230"}
Jul 09 14:07:00.441310 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:00.441268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv" event={"ID":"38858016-310b-44cb-b5be-957a055a2e35","Type":"ContainerStarted","Data":"c0ba2ed05bc8aab7bccd7d5ff1746f3cfb793437d761832628e28eb6513ebc4f"}
Jul 09 14:07:00.441755 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:00.441384    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:07:00.457209 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:00.457152    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv" podStartSLOduration=1.723601669 podStartE2EDuration="6.457135504s" totalImagesPullingTime="4.733533835s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:00.455809955 +0000 UTC m=+525.997076764" watchObservedRunningTime="2026-07-09 14:07:00.457135504 +0000 UTC m=+525.998402314"
Jul 09 14:07:06.482376 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:06.482296    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-9dbcv"
Jul 09 14:07:07.034763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.034718    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:07.039791 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.039410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:07.044160 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.044138    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:07.120724 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.120682    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2v7wk\" (UniqueName: \"kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk\") pod \"authorino-5d4b67c446-mnw4f\" (UID: \"ea96b193-28f2-4ba3-976b-bd5cdea3b45a\") " pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:07.222016 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.221972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2v7wk\" (UniqueName: \"kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk\") pod \"authorino-5d4b67c446-mnw4f\" (UID: \"ea96b193-28f2-4ba3-976b-bd5cdea3b45a\") " pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:07.230089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.230058    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2v7wk\" (UniqueName: \"kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk\") pod \"authorino-5d4b67c446-mnw4f\" (UID: \"ea96b193-28f2-4ba3-976b-bd5cdea3b45a\") " pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:07.267544 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.267503    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:07.267819 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.267805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:07.293442 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.293369    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-668cc88cdb-zxvzq"]
Jul 09 14:07:07.296923 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.296902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.303990 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.303960    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-668cc88cdb-zxvzq"]
Jul 09 14:07:07.379891 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.379857    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-668cc88cdb-zxvzq"]
Jul 09 14:07:07.380147 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:07:07.380125    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-jvsdv], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-668cc88cdb-zxvzq" podUID="89c60969-ae06-4ac5-86b2-32db20e857d2"
Jul 09 14:07:07.405562 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.405527    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:07:07.409238 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.409214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.410064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.410040    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:07.411754 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.411733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 14:07:07.413611 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:07.413562    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea96b193_28f2_4ba3_976b_bd5cdea3b45a.slice/crio-869a96278b36f149304405af671138b70268af2ceb6d98aa23157e6f968b2b89 WatchSource:0}: Error finding container 869a96278b36f149304405af671138b70268af2ceb6d98aa23157e6f968b2b89: Status 404 returned error can't find the container with id 869a96278b36f149304405af671138b70268af2ceb6d98aa23157e6f968b2b89
Jul 09 14:07:07.418394 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.418372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:07:07.424399 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.424367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jvsdv\" (UniqueName: \"kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv\") pod \"authorino-668cc88cdb-zxvzq\" (UID: \"89c60969-ae06-4ac5-86b2-32db20e857d2\") " pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.478591 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.478539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" event={"ID":"ea96b193-28f2-4ba3-976b-bd5cdea3b45a","Type":"ContainerStarted","Data":"869a96278b36f149304405af671138b70268af2ceb6d98aa23157e6f968b2b89"}
Jul 09 14:07:07.478591 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.478581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.485295 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.485271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.525672 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.525631    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jvsdv\" (UniqueName: \"kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv\") pod \"authorino-668cc88cdb-zxvzq\" (UID: \"89c60969-ae06-4ac5-86b2-32db20e857d2\") " pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.525672 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.525685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpcc8\" (UniqueName: \"kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.525935 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.525769    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.534103 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.534078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jvsdv\" (UniqueName: \"kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv\") pod \"authorino-668cc88cdb-zxvzq\" (UID: \"89c60969-ae06-4ac5-86b2-32db20e857d2\") " pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:07.626373 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.626333    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jvsdv\" (UniqueName: \"kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv\") pod \"89c60969-ae06-4ac5-86b2-32db20e857d2\" (UID: \"89c60969-ae06-4ac5-86b2-32db20e857d2\") "
Jul 09 14:07:07.626599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.626500    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpcc8\" (UniqueName: \"kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.626663 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.626600    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.628657 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.628624    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv" (OuterVolumeSpecName: "kube-api-access-jvsdv") pod "89c60969-ae06-4ac5-86b2-32db20e857d2" (UID: "89c60969-ae06-4ac5-86b2-32db20e857d2"). InnerVolumeSpecName "kube-api-access-jvsdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:07.629005 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.628988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.634409 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.634383    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpcc8\" (UniqueName: \"kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8\") pod \"authorino-bf6446ddb-xqh7f\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") " pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.727374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.727342    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jvsdv\" (UniqueName: \"kubernetes.io/projected/89c60969-ae06-4ac5-86b2-32db20e857d2-kube-api-access-jvsdv\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:07:07.728166 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.728144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:07:07.872168 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:07.872133    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:07:07.875266 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:07.875237    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b6aea47_78f5_4761_a241_9b55cfc6b635.slice/crio-36e77a256ddefa0fd61b02e6c988d185d2f0e2685071fa45421b0e387fb7a7e6 WatchSource:0}: Error finding container 36e77a256ddefa0fd61b02e6c988d185d2f0e2685071fa45421b0e387fb7a7e6: Status 404 returned error can't find the container with id 36e77a256ddefa0fd61b02e6c988d185d2f0e2685071fa45421b0e387fb7a7e6
Jul 09 14:07:08.486065 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.486030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" event={"ID":"2b6aea47-78f5-4761-a241-9b55cfc6b635","Type":"ContainerStarted","Data":"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"}
Jul 09 14:07:08.486524 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.486080    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" event={"ID":"2b6aea47-78f5-4761-a241-9b55cfc6b635","Type":"ContainerStarted","Data":"36e77a256ddefa0fd61b02e6c988d185d2f0e2685071fa45421b0e387fb7a7e6"}
Jul 09 14:07:08.487603 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.487553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-668cc88cdb-zxvzq"
Jul 09 14:07:08.487754 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.487601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" event={"ID":"ea96b193-28f2-4ba3-976b-bd5cdea3b45a","Type":"ContainerStarted","Data":"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"}
Jul 09 14:07:08.487754 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.487633    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" podUID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" containerName="authorino" containerID="cri-o://4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c" gracePeriod=30
Jul 09 14:07:08.502925 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.502877    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" podStartSLOduration=1.129154476 podStartE2EDuration="1.502859754s" totalImagesPullingTime="373.705278ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:08.50110655 +0000 UTC m=+534.042373334" watchObservedRunningTime="2026-07-09 14:07:08.502859754 +0000 UTC m=+534.044126536"
Jul 09 14:07:08.517237 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.517181    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" podStartSLOduration=1.119981719 podStartE2EDuration="1.517164286s" totalImagesPullingTime="397.182567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:08.515737319 +0000 UTC m=+534.057004103" watchObservedRunningTime="2026-07-09 14:07:08.517164286 +0000 UTC m=+534.058431068"
Jul 09 14:07:08.528643 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.528609    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:07:08.528825 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.528803    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-hxq5p" podUID="0dc079c0-54ba-433d-a4eb-4f9be563e965" containerName="authorino" containerID="cri-o://929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546" gracePeriod=30
Jul 09 14:07:08.542611 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.542563    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-668cc88cdb-zxvzq"]
Jul 09 14:07:08.548666 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.548633    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-668cc88cdb-zxvzq"]
Jul 09 14:07:08.803142 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.803113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:08.815404 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.815379    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:07:08.936762 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.936721    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hqr6x\" (UniqueName: \"kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x\") pod \"0dc079c0-54ba-433d-a4eb-4f9be563e965\" (UID: \"0dc079c0-54ba-433d-a4eb-4f9be563e965\") "
Jul 09 14:07:08.936941 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.936808    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2v7wk\" (UniqueName: \"kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk\") pod \"ea96b193-28f2-4ba3-976b-bd5cdea3b45a\" (UID: \"ea96b193-28f2-4ba3-976b-bd5cdea3b45a\") "
Jul 09 14:07:08.939098 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.939061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk" (OuterVolumeSpecName: "kube-api-access-2v7wk") pod "ea96b193-28f2-4ba3-976b-bd5cdea3b45a" (UID: "ea96b193-28f2-4ba3-976b-bd5cdea3b45a"). InnerVolumeSpecName "kube-api-access-2v7wk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:08.939221 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:08.939061    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x" (OuterVolumeSpecName: "kube-api-access-hqr6x") pod "0dc079c0-54ba-433d-a4eb-4f9be563e965" (UID: "0dc079c0-54ba-433d-a4eb-4f9be563e965"). InnerVolumeSpecName "kube-api-access-hqr6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:09.038347 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.038301    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2v7wk\" (UniqueName: \"kubernetes.io/projected/ea96b193-28f2-4ba3-976b-bd5cdea3b45a-kube-api-access-2v7wk\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:07:09.038347 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.038340    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hqr6x\" (UniqueName: \"kubernetes.io/projected/0dc079c0-54ba-433d-a4eb-4f9be563e965-kube-api-access-hqr6x\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:07:09.062103 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.062068    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89c60969-ae06-4ac5-86b2-32db20e857d2" path="/var/lib/kubelet/pods/89c60969-ae06-4ac5-86b2-32db20e857d2/volumes"
Jul 09 14:07:09.499008 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.498915    2580 generic.go:356] "Generic (PLEG): container finished" podID="0dc079c0-54ba-433d-a4eb-4f9be563e965" containerID="929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546" exitCode=0
Jul 09 14:07:09.499008 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.498980    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-hxq5p"
Jul 09 14:07:09.499494 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.499003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hxq5p" event={"ID":"0dc079c0-54ba-433d-a4eb-4f9be563e965","Type":"ContainerDied","Data":"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"}
Jul 09 14:07:09.499494 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.499062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-hxq5p" event={"ID":"0dc079c0-54ba-433d-a4eb-4f9be563e965","Type":"ContainerDied","Data":"68ff1194fa7a0307bf86a97c51132b493202530b04021eaaa80b2e6f5d0e3ac1"}
Jul 09 14:07:09.499494 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.499087    2580 scope.go:117] "RemoveContainer" containerID="929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"
Jul 09 14:07:09.501020 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.500907    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" containerID="4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c" exitCode=0
Jul 09 14:07:09.501020 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.500985    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-mnw4f"
Jul 09 14:07:09.501020 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.500999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" event={"ID":"ea96b193-28f2-4ba3-976b-bd5cdea3b45a","Type":"ContainerDied","Data":"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"}
Jul 09 14:07:09.501220 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.501049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-mnw4f" event={"ID":"ea96b193-28f2-4ba3-976b-bd5cdea3b45a","Type":"ContainerDied","Data":"869a96278b36f149304405af671138b70268af2ceb6d98aa23157e6f968b2b89"}
Jul 09 14:07:09.508341 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.508321    2580 scope.go:117] "RemoveContainer" containerID="929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"
Jul 09 14:07:09.508731 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:07:09.508709    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546\": container with ID starting with 929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546 not found: ID does not exist" containerID="929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"
Jul 09 14:07:09.508847 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.508741    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546"} err="failed to get container status \"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546\": rpc error: code = NotFound desc = could not find container \"929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546\": container with ID starting with 929695a2c925ce01c6883169b26607a8c8cf298f5234c4243b631bd6b4265546 not found: ID does not exist"
Jul 09 14:07:09.508847 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.508761    2580 scope.go:117] "RemoveContainer" containerID="4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"
Jul 09 14:07:09.517270 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.517249    2580 scope.go:117] "RemoveContainer" containerID="4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"
Jul 09 14:07:09.517712 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:07:09.517672    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c\": container with ID starting with 4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c not found: ID does not exist" containerID="4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"
Jul 09 14:07:09.517853 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.517723    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c"} err="failed to get container status \"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c\": rpc error: code = NotFound desc = could not find container \"4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c\": container with ID starting with 4a1196e8a0ca64c13ed34a3aea36803072eb08286a6275f3ce7460b05bcb7a5c not found: ID does not exist"
Jul 09 14:07:09.518802 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.518782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:09.522322 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.522298    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-mnw4f"]
Jul 09 14:07:09.534002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.533975    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:07:09.536064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:09.536038    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-hxq5p"]
Jul 09 14:07:11.059832 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:11.059798    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0dc079c0-54ba-433d-a4eb-4f9be563e965" path="/var/lib/kubelet/pods/0dc079c0-54ba-433d-a4eb-4f9be563e965/volumes"
Jul 09 14:07:11.060301 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:11.060113    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" path="/var/lib/kubelet/pods/ea96b193-28f2-4ba3-976b-bd5cdea3b45a/volumes"
Jul 09 14:07:13.257751 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.257711    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258045    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0dc079c0-54ba-433d-a4eb-4f9be563e965" containerName="authorino"
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258057    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0dc079c0-54ba-433d-a4eb-4f9be563e965" containerName="authorino"
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258069    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" containerName="authorino"
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258075    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" containerName="authorino"
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258144    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea96b193-28f2-4ba3-976b-bd5cdea3b45a" containerName="authorino"
Jul 09 14:07:13.258167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.258155    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0dc079c0-54ba-433d-a4eb-4f9be563e965" containerName="authorino"
Jul 09 14:07:13.443517 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.443476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:13.443517 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.443525    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:07:13.443767 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.443612    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.447261 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.447212    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 14:07:13.447261 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.447231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ln8rm\""
Jul 09 14:07:13.447261 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.447231    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 14:07:13.448000 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.447982    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.456726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.456698    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:07:13.579684 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.579653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.579876 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.579700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jl5j\" (UniqueName: \"kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.579876 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.579725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.579876 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.579836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8llxt\" (UniqueName: \"kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.680524 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.680485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.680524 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.680529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jl5j\" (UniqueName: \"kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.680783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.680551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.680783 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.680633    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8llxt\" (UniqueName: \"kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.683277 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.683248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.683484 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.683463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.689091 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.689063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8llxt\" (UniqueName: \"kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt\") pod \"maas-controller-849cbdf4c4-vlzws\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") " pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.689224 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.689180    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jl5j\" (UniqueName: \"kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j\") pod \"maas-controller-6cf4cbf58c-zl4cf\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") " pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.757278 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.757234    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:13.765418 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.765378    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:13.966820 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.966788    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:13.969142 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:13.969108    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1cb6e37_1361_4848_8f9f_25fa1f530362.slice/crio-41f3205f7332f5872c371eb42e05b48ef80d6e58d8ad8f31d01948b5dafeb629 WatchSource:0}: Error finding container 41f3205f7332f5872c371eb42e05b48ef80d6e58d8ad8f31d01948b5dafeb629: Status 404 returned error can't find the container with id 41f3205f7332f5872c371eb42e05b48ef80d6e58d8ad8f31d01948b5dafeb629
Jul 09 14:07:13.994300 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:13.994274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:07:13.996824 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:13.996793    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac44ee9_7b68_4e39_a23b_71e78c2a085c.slice/crio-74abb6807c547028a1246e9fec0fc45bf43d2f9c2eae630ed2560a76a64e85f6 WatchSource:0}: Error finding container 74abb6807c547028a1246e9fec0fc45bf43d2f9c2eae630ed2560a76a64e85f6: Status 404 returned error can't find the container with id 74abb6807c547028a1246e9fec0fc45bf43d2f9c2eae630ed2560a76a64e85f6
Jul 09 14:07:14.528222 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:14.528182    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" event={"ID":"b1cb6e37-1361-4848-8f9f-25fa1f530362","Type":"ContainerStarted","Data":"41f3205f7332f5872c371eb42e05b48ef80d6e58d8ad8f31d01948b5dafeb629"}
Jul 09 14:07:14.529374 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:14.529346    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" event={"ID":"eac44ee9-7b68-4e39-a23b-71e78c2a085c","Type":"ContainerStarted","Data":"74abb6807c547028a1246e9fec0fc45bf43d2f9c2eae630ed2560a76a64e85f6"}
Jul 09 14:07:17.486440 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:17.486413    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 14:07:18.552748 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.552703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" event={"ID":"b1cb6e37-1361-4848-8f9f-25fa1f530362","Type":"ContainerStarted","Data":"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"}
Jul 09 14:07:18.553230 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.552887    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:18.554205 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.554180    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" event={"ID":"eac44ee9-7b68-4e39-a23b-71e78c2a085c","Type":"ContainerStarted","Data":"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"}
Jul 09 14:07:18.554339 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.554324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:18.571412 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.571364    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" podStartSLOduration=2.048999331 podStartE2EDuration="5.571347801s" totalImagesPullingTime="3.52234847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:18.569772323 +0000 UTC m=+544.111039128" watchObservedRunningTime="2026-07-09 14:07:18.571347801 +0000 UTC m=+544.112614583"
Jul 09 14:07:18.586161 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:18.586110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" podStartSLOduration=2.100288178 podStartE2EDuration="5.586094667s" totalImagesPullingTime="3.485806489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:18.584694092 +0000 UTC m=+544.125960899" watchObservedRunningTime="2026-07-09 14:07:18.586094667 +0000 UTC m=+544.127361478"
Jul 09 14:07:29.565318 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.565279    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:07:29.565789 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.565354    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:29.627014 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.626972    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:29.627206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.627177    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" podUID="b1cb6e37-1361-4848-8f9f-25fa1f530362" containerName="manager" containerID="cri-o://340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0" gracePeriod=30
Jul 09 14:07:29.875536 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.875503    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:29.932984 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.932944    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert\") pod \"b1cb6e37-1361-4848-8f9f-25fa1f530362\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") "
Jul 09 14:07:29.933168 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.933059    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jl5j\" (UniqueName: \"kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j\") pod \"b1cb6e37-1361-4848-8f9f-25fa1f530362\" (UID: \"b1cb6e37-1361-4848-8f9f-25fa1f530362\") "
Jul 09 14:07:29.935518 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.935473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b1cb6e37-1361-4848-8f9f-25fa1f530362" (UID: "b1cb6e37-1361-4848-8f9f-25fa1f530362"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:07:29.935518 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:29.935490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j" (OuterVolumeSpecName: "kube-api-access-2jl5j") pod "b1cb6e37-1361-4848-8f9f-25fa1f530362" (UID: "b1cb6e37-1361-4848-8f9f-25fa1f530362"). InnerVolumeSpecName "kube-api-access-2jl5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:07:30.033800 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.033756    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jl5j\" (UniqueName: \"kubernetes.io/projected/b1cb6e37-1361-4848-8f9f-25fa1f530362-kube-api-access-2jl5j\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:07:30.033800 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.033790    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b1cb6e37-1361-4848-8f9f-25fa1f530362-webhook-cert\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:07:30.608771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.608726    2580 generic.go:356] "Generic (PLEG): container finished" podID="b1cb6e37-1361-4848-8f9f-25fa1f530362" containerID="340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0" exitCode=0
Jul 09 14:07:30.608771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.608773    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf"
Jul 09 14:07:30.609330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.608775    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" event={"ID":"b1cb6e37-1361-4848-8f9f-25fa1f530362","Type":"ContainerDied","Data":"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"}
Jul 09 14:07:30.609330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.608845    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6cf4cbf58c-zl4cf" event={"ID":"b1cb6e37-1361-4848-8f9f-25fa1f530362","Type":"ContainerDied","Data":"41f3205f7332f5872c371eb42e05b48ef80d6e58d8ad8f31d01948b5dafeb629"}
Jul 09 14:07:30.609330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.608876    2580 scope.go:117] "RemoveContainer" containerID="340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"
Jul 09 14:07:30.618248 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.618229    2580 scope.go:117] "RemoveContainer" containerID="340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"
Jul 09 14:07:30.618591 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:07:30.618551    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0\": container with ID starting with 340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0 not found: ID does not exist" containerID="340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"
Jul 09 14:07:30.618646 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.618597    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0"} err="failed to get container status \"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0\": rpc error: code = NotFound desc = could not find container \"340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0\": container with ID starting with 340ebe45906ce4867f8ddfbae311d9b87ccb4ab3258865df233271cceddea6d0 not found: ID does not exist"
Jul 09 14:07:30.631461 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.631418    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:30.637676 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:30.634450    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6cf4cbf58c-zl4cf"]
Jul 09 14:07:31.059994 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:31.059959    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1cb6e37-1361-4848-8f9f-25fa1f530362" path="/var/lib/kubelet/pods/b1cb6e37-1361-4848-8f9f-25fa1f530362/volumes"
Jul 09 14:07:50.904849 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.904805    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"]
Jul 09 14:07:50.905362 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.905342    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1cb6e37-1361-4848-8f9f-25fa1f530362" containerName="manager"
Jul 09 14:07:50.905410 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.905366    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1cb6e37-1361-4848-8f9f-25fa1f530362" containerName="manager"
Jul 09 14:07:50.905450 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.905440    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1cb6e37-1361-4848-8f9f-25fa1f530362" containerName="manager"
Jul 09 14:07:50.909972 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.909947    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:50.912535 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.912506    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-5r4q9\""
Jul 09 14:07:50.912712 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.912650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 14:07:50.925404 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:50.925365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"]
Jul 09 14:07:51.014738 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.014703    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-maas-api-tls\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.014738 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.014743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzsp9\" (UniqueName: \"kubernetes.io/projected/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-kube-api-access-fzsp9\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.014964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.014832    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-openshift-service-ca\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.116125 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.116083    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-maas-api-tls\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.116125 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.116122    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzsp9\" (UniqueName: \"kubernetes.io/projected/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-kube-api-access-fzsp9\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.116408 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.116166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-openshift-service-ca\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.116867 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.116846    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-openshift-service-ca\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.118789 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.118763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-maas-api-tls\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.127227 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.127202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzsp9\" (UniqueName: \"kubernetes.io/projected/c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d-kube-api-access-fzsp9\") pod \"maas-api-74b776d86c-f4gnb\" (UID: \"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d\") " pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.225306 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.225214    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:51.371511 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.371471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"]
Jul 09 14:07:51.374787 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:51.374760    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0dbf1f8_de6a_4f9e_93cd_75be3d710f1d.slice/crio-95730c921a4d75c8d21dc79aa99daa72b2297c86a7cbf209c6e3f466c3cc2ff5 WatchSource:0}: Error finding container 95730c921a4d75c8d21dc79aa99daa72b2297c86a7cbf209c6e3f466c3cc2ff5: Status 404 returned error can't find the container with id 95730c921a4d75c8d21dc79aa99daa72b2297c86a7cbf209c6e3f466c3cc2ff5
Jul 09 14:07:51.482349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.482265    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ldp5q"]
Jul 09 14:07:51.485029 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.485010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.487457 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.487433    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mvvl2\""
Jul 09 14:07:51.487703 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.487442    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 14:07:51.494061 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.494033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ldp5q"]
Jul 09 14:07:51.519698 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.519660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f987c4fe-cd77-4961-93c2-3b0166f8b67c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.519864 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.519717    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hws92\" (UniqueName: \"kubernetes.io/projected/f987c4fe-cd77-4961-93c2-3b0166f8b67c-kube-api-access-hws92\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.621047 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.621011    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hws92\" (UniqueName: \"kubernetes.io/projected/f987c4fe-cd77-4961-93c2-3b0166f8b67c-kube-api-access-hws92\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.621220 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.621084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f987c4fe-cd77-4961-93c2-3b0166f8b67c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.621756 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.621739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f987c4fe-cd77-4961-93c2-3b0166f8b67c-plugins-config-volume\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.629638 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.629612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hws92\" (UniqueName: \"kubernetes.io/projected/f987c4fe-cd77-4961-93c2-3b0166f8b67c-kube-api-access-hws92\") pod \"payload-pre-processing-7964457696-ldp5q\" (UID: \"f987c4fe-cd77-4961-93c2-3b0166f8b67c\") " pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.705681 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.705641    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb" event={"ID":"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d","Type":"ContainerStarted","Data":"95730c921a4d75c8d21dc79aa99daa72b2297c86a7cbf209c6e3f466c3cc2ff5"}
Jul 09 14:07:51.796905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.796799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:51.938870 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:51.938842    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-ldp5q"]
Jul 09 14:07:51.941391 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:07:51.941366    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf987c4fe_cd77_4961_93c2_3b0166f8b67c.slice/crio-c559b201d2bfe0e25518092a8114170e1262798c456e0d2987815f7acdb6c408 WatchSource:0}: Error finding container c559b201d2bfe0e25518092a8114170e1262798c456e0d2987815f7acdb6c408: Status 404 returned error can't find the container with id c559b201d2bfe0e25518092a8114170e1262798c456e0d2987815f7acdb6c408
Jul 09 14:07:52.712002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:52.711963    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q" event={"ID":"f987c4fe-cd77-4961-93c2-3b0166f8b67c","Type":"ContainerStarted","Data":"c559b201d2bfe0e25518092a8114170e1262798c456e0d2987815f7acdb6c408"}
Jul 09 14:07:53.720376 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:53.720285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q" event={"ID":"f987c4fe-cd77-4961-93c2-3b0166f8b67c","Type":"ContainerStarted","Data":"db1003771aa9bd15a2ced643d6eea6d3edd464fb42c5807cf613cc83e7237786"}
Jul 09 14:07:53.720842 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:53.720449    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:07:54.731213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:54.731151    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb" event={"ID":"c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d","Type":"ContainerStarted","Data":"a6a205cad266039d0a4987c69200ed9b2ba72044897957a655774668455fa90b"}
Jul 09 14:07:54.731700 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:54.731405    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:07:54.747985 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:54.747929    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb" podStartSLOduration=1.824974289 podStartE2EDuration="4.747908974s" totalImagesPullingTime="2.922934685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:54.747457271 +0000 UTC m=+580.288724069" watchObservedRunningTime="2026-07-09 14:07:54.747908974 +0000 UTC m=+580.289175758"
Jul 09 14:07:54.749168 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:07:54.749137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q" podStartSLOduration=2.281855181 podStartE2EDuration="3.749127248s" totalImagesPullingTime="1.467272067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:07:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:07:53.736146161 +0000 UTC m=+579.277412966" watchObservedRunningTime="2026-07-09 14:07:54.749127248 +0000 UTC m=+580.290394031"
Jul 09 14:08:00.741368 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:00.741336    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-74b776d86c-f4gnb"
Jul 09 14:08:04.733200 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:04.733172    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-ldp5q"
Jul 09 14:08:14.912434 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:14.912403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:08:14.918458 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:14.918431    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:08:14.919367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:14.919348    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:08:14.925111 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:14.925091    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:08:26.634519 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.634476    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"]
Jul 09 14:08:26.639362 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.639343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.644584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.644544    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 14:08:26.645028 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.645010    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 14:08:26.645473 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.645448    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-cjpmt\""
Jul 09 14:08:26.645615 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.645590    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 14:08:26.649896 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.649872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"]
Jul 09 14:08:26.738003 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.737971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.738159 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.738045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.738159 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.738089    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.738159 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.738116    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d1b60-2ab5-430c-99b7-48943f74c529-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.738159 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.738142    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdrfv\" (UniqueName: \"kubernetes.io/projected/701d1b60-2ab5-430c-99b7-48943f74c529-kube-api-access-bdrfv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.738317 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.738168    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839463 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839463 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d1b60-2ab5-430c-99b7-48943f74c529-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdrfv\" (UniqueName: \"kubernetes.io/projected/701d1b60-2ab5-430c-99b7-48943f74c529-kube-api-access-bdrfv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839881 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.839957 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.840014 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.839994    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.840102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.840082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.842036 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.842012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/701d1b60-2ab5-430c-99b7-48943f74c529-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.842176 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.842158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/701d1b60-2ab5-430c-99b7-48943f74c529-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.847261 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.847239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdrfv\" (UniqueName: \"kubernetes.io/projected/701d1b60-2ab5-430c-99b7-48943f74c529-kube-api-access-bdrfv\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qwf6k\" (UID: \"701d1b60-2ab5-430c-99b7-48943f74c529\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:26.956436 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:26.956353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:27.091697 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:27.091673    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"]
Jul 09 14:08:27.094357 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:08:27.094330    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod701d1b60_2ab5_430c_99b7_48943f74c529.slice/crio-35d43699b6c8216fc3ab36875dceec789474a365447850eb805c6ff907eab0ea WatchSource:0}: Error finding container 35d43699b6c8216fc3ab36875dceec789474a365447850eb805c6ff907eab0ea: Status 404 returned error can't find the container with id 35d43699b6c8216fc3ab36875dceec789474a365447850eb805c6ff907eab0ea
Jul 09 14:08:27.886698 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:27.886621    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k" event={"ID":"701d1b60-2ab5-430c-99b7-48943f74c529","Type":"ContainerStarted","Data":"35d43699b6c8216fc3ab36875dceec789474a365447850eb805c6ff907eab0ea"}
Jul 09 14:08:28.893224 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:28.893137    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k" event={"ID":"701d1b60-2ab5-430c-99b7-48943f74c529","Type":"ContainerStarted","Data":"f1e4b3dc0863ab629260f7a41205f55fae31a62b56f24755435c9b73e0f8b9b0"}
Jul 09 14:08:28.910451 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:28.910392    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k" podStartSLOduration=1.436283433 podStartE2EDuration="2.910374011s" totalImagesPullingTime="1.474090578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:28.909070489 +0000 UTC m=+614.450337273" watchObservedRunningTime="2026-07-09 14:08:28.910374011 +0000 UTC m=+614.451640794"
Jul 09 14:08:36.957483 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:36.957385    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:36.957483 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:36.957460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:36.970540 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:36.970512    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:37.946143 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:37.946109    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qwf6k"
Jul 09 14:08:41.011800 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.011770    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"]
Jul 09 14:08:41.015127 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.015106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.017434 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.017416    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 09 14:08:41.023643 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.023621    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"]
Jul 09 14:08:41.072304 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072266    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.072497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.072497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.072497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.072497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.072497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.072448    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsr7d\" (UniqueName: \"kubernetes.io/projected/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-kube-api-access-jsr7d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173619 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsr7d\" (UniqueName: \"kubernetes.io/projected/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-kube-api-access-jsr7d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173649    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.173817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.173762    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.174148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.174128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.174195 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.174157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.174195 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.174135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.176209 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.176177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.176436 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.176420    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.182129 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.182104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsr7d\" (UniqueName: \"kubernetes.io/projected/4ae68b94-ee40-4c53-97f8-2a61a208ca8e-kube-api-access-jsr7d\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5\" (UID: \"4ae68b94-ee40-4c53-97f8-2a61a208ca8e\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.326990 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.326957    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:41.461309 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.461285    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"]
Jul 09 14:08:41.463241 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:08:41.463197    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae68b94_ee40_4c53_97f8_2a61a208ca8e.slice/crio-466027f625652afc822fb2ec4c8f75094937c8a58db5698e357cbaad9f4304e2 WatchSource:0}: Error finding container 466027f625652afc822fb2ec4c8f75094937c8a58db5698e357cbaad9f4304e2: Status 404 returned error can't find the container with id 466027f625652afc822fb2ec4c8f75094937c8a58db5698e357cbaad9f4304e2
Jul 09 14:08:41.464554 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.464533    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:08:41.953579 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.953540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5" event={"ID":"4ae68b94-ee40-4c53-97f8-2a61a208ca8e","Type":"ContainerStarted","Data":"b84d0576de99ab7c00bbd262eae14e24ca2e0bee7b5e53e590a6ec7dfffdf5f7"}
Jul 09 14:08:41.953772 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.953619    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5" event={"ID":"4ae68b94-ee40-4c53-97f8-2a61a208ca8e","Type":"ContainerStarted","Data":"466027f625652afc822fb2ec4c8f75094937c8a58db5698e357cbaad9f4304e2"}
Jul 09 14:08:41.973358 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:41.973293    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5" podStartSLOduration=1.649662675 podStartE2EDuration="1.973273074s" totalImagesPullingTime="323.610399ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:08:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:08:41.973036444 +0000 UTC m=+627.514303261" watchObservedRunningTime="2026-07-09 14:08:41.973273074 +0000 UTC m=+627.514539858"
Jul 09 14:08:51.327777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:51.327714    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:51.328165 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:51.327797    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:51.340399 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:51.340372    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:08:52.009584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:08:52.009546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5"
Jul 09 14:10:04.388562 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.388526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-74c6c5b44c-p22ln"]
Jul 09 14:10:04.392111 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.392095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.399701 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.399675    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74c6c5b44c-p22ln"]
Jul 09 14:10:04.432872 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.432826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmrgc\" (UniqueName: \"kubernetes.io/projected/ea1c04c7-2027-417e-bb7e-de219d89e06b-kube-api-access-qmrgc\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.433055 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.432891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea1c04c7-2027-417e-bb7e-de219d89e06b-tls-cert\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.533730 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.533668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmrgc\" (UniqueName: \"kubernetes.io/projected/ea1c04c7-2027-417e-bb7e-de219d89e06b-kube-api-access-qmrgc\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.533921 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.533761    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea1c04c7-2027-417e-bb7e-de219d89e06b-tls-cert\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.536611 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.536557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/ea1c04c7-2027-417e-bb7e-de219d89e06b-tls-cert\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.542503 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.542471    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmrgc\" (UniqueName: \"kubernetes.io/projected/ea1c04c7-2027-417e-bb7e-de219d89e06b-kube-api-access-qmrgc\") pod \"authorino-74c6c5b44c-p22ln\" (UID: \"ea1c04c7-2027-417e-bb7e-de219d89e06b\") " pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.702291 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.702199    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-74c6c5b44c-p22ln"
Jul 09 14:10:04.841119 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:04.841078    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-74c6c5b44c-p22ln"]
Jul 09 14:10:04.848123 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:10:04.848095    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea1c04c7_2027_417e_bb7e_de219d89e06b.slice/crio-895c32bee4d35f7707251f26cb258dc346ec9bcfe0e26cbb73b2d61734e03116 WatchSource:0}: Error finding container 895c32bee4d35f7707251f26cb258dc346ec9bcfe0e26cbb73b2d61734e03116: Status 404 returned error can't find the container with id 895c32bee4d35f7707251f26cb258dc346ec9bcfe0e26cbb73b2d61734e03116
Jul 09 14:10:05.334772 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:05.334722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74c6c5b44c-p22ln" event={"ID":"ea1c04c7-2027-417e-bb7e-de219d89e06b","Type":"ContainerStarted","Data":"895c32bee4d35f7707251f26cb258dc346ec9bcfe0e26cbb73b2d61734e03116"}
Jul 09 14:10:07.348452 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.348358    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-74c6c5b44c-p22ln" event={"ID":"ea1c04c7-2027-417e-bb7e-de219d89e06b","Type":"ContainerStarted","Data":"48ace43f77a91eae9b18d0126fce299913505f6b665a16e2cd791ca259e37401"}
Jul 09 14:10:07.365688 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.365315    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-74c6c5b44c-p22ln" podStartSLOduration=1.148547399 podStartE2EDuration="3.365291555s" totalImagesPullingTime="2.216744156s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:10:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:10:07.364275984 +0000 UTC m=+712.905542768" watchObservedRunningTime="2026-07-09 14:10:07.365291555 +0000 UTC m=+712.906558340"
Jul 09 14:10:07.393591 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.393528    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:10:07.393853 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.393818    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" podUID="2b6aea47-78f5-4761-a241-9b55cfc6b635" containerName="authorino" containerID="cri-o://05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c" gracePeriod=30
Jul 09 14:10:07.646515 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.646491    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:10:07.657448 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.656033    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert\") pod \"2b6aea47-78f5-4761-a241-9b55cfc6b635\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") "
Jul 09 14:10:07.657448 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.656170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpcc8\" (UniqueName: \"kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8\") pod \"2b6aea47-78f5-4761-a241-9b55cfc6b635\" (UID: \"2b6aea47-78f5-4761-a241-9b55cfc6b635\") "
Jul 09 14:10:07.664321 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.664281    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8" (OuterVolumeSpecName: "kube-api-access-vpcc8") pod "2b6aea47-78f5-4761-a241-9b55cfc6b635" (UID: "2b6aea47-78f5-4761-a241-9b55cfc6b635"). InnerVolumeSpecName "kube-api-access-vpcc8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:10:07.673373 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.673333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "2b6aea47-78f5-4761-a241-9b55cfc6b635" (UID: "2b6aea47-78f5-4761-a241-9b55cfc6b635"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:10:07.757743 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.757697    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2b6aea47-78f5-4761-a241-9b55cfc6b635-tls-cert\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:10:07.757743 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:07.757737    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpcc8\" (UniqueName: \"kubernetes.io/projected/2b6aea47-78f5-4761-a241-9b55cfc6b635-kube-api-access-vpcc8\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:10:08.354814 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.354722    2580 generic.go:356] "Generic (PLEG): container finished" podID="2b6aea47-78f5-4761-a241-9b55cfc6b635" containerID="05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c" exitCode=0
Jul 09 14:10:08.354814 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.354770    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bf6446ddb-xqh7f"
Jul 09 14:10:08.355628 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.354822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" event={"ID":"2b6aea47-78f5-4761-a241-9b55cfc6b635","Type":"ContainerDied","Data":"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"}
Jul 09 14:10:08.355628 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.354876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bf6446ddb-xqh7f" event={"ID":"2b6aea47-78f5-4761-a241-9b55cfc6b635","Type":"ContainerDied","Data":"36e77a256ddefa0fd61b02e6c988d185d2f0e2685071fa45421b0e387fb7a7e6"}
Jul 09 14:10:08.355628 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.354901    2580 scope.go:117] "RemoveContainer" containerID="05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"
Jul 09 14:10:08.364367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.364348    2580 scope.go:117] "RemoveContainer" containerID="05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"
Jul 09 14:10:08.364686 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:10:08.364662    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c\": container with ID starting with 05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c not found: ID does not exist" containerID="05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"
Jul 09 14:10:08.364756 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.364695    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c"} err="failed to get container status \"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c\": rpc error: code = NotFound desc = could not find container \"05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c\": container with ID starting with 05f59678dc2ecbf016fa02ffad198efc5617670b5c773b6ee684c181f705d58c not found: ID does not exist"
Jul 09 14:10:08.376686 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.376651    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:10:08.380501 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:08.380471    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-bf6446ddb-xqh7f"]
Jul 09 14:10:09.061062 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:10:09.061016    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2b6aea47-78f5-4761-a241-9b55cfc6b635" path="/var/lib/kubelet/pods/2b6aea47-78f5-4761-a241-9b55cfc6b635/volumes"
Jul 09 14:11:20.142937 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.142898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:11:20.143504 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.143154    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" podUID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" containerName="manager" containerID="cri-o://b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4" gracePeriod=30
Jul 09 14:11:20.391439 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.391414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:11:20.536858 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.536755    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8llxt\" (UniqueName: \"kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt\") pod \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") "
Jul 09 14:11:20.536858 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.536817    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert\") pod \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\" (UID: \"eac44ee9-7b68-4e39-a23b-71e78c2a085c\") "
Jul 09 14:11:20.539116 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.539076    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "eac44ee9-7b68-4e39-a23b-71e78c2a085c" (UID: "eac44ee9-7b68-4e39-a23b-71e78c2a085c"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:11:20.539256 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.539082    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt" (OuterVolumeSpecName: "kube-api-access-8llxt") pod "eac44ee9-7b68-4e39-a23b-71e78c2a085c" (UID: "eac44ee9-7b68-4e39-a23b-71e78c2a085c"). InnerVolumeSpecName "kube-api-access-8llxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:11:20.638445 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.638403    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eac44ee9-7b68-4e39-a23b-71e78c2a085c-webhook-cert\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:11:20.638445 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.638439    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8llxt\" (UniqueName: \"kubernetes.io/projected/eac44ee9-7b68-4e39-a23b-71e78c2a085c-kube-api-access-8llxt\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:11:20.699359 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.699327    2580 generic.go:356] "Generic (PLEG): container finished" podID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" containerID="b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4" exitCode=0
Jul 09 14:11:20.699531 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.699375    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-vlzws"
Jul 09 14:11:20.699531 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.699412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" event={"ID":"eac44ee9-7b68-4e39-a23b-71e78c2a085c","Type":"ContainerDied","Data":"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"}
Jul 09 14:11:20.699531 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.699457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-vlzws" event={"ID":"eac44ee9-7b68-4e39-a23b-71e78c2a085c","Type":"ContainerDied","Data":"74abb6807c547028a1246e9fec0fc45bf43d2f9c2eae630ed2560a76a64e85f6"}
Jul 09 14:11:20.699531 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.699481    2580 scope.go:117] "RemoveContainer" containerID="b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"
Jul 09 14:11:20.708585 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.708553    2580 scope.go:117] "RemoveContainer" containerID="b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"
Jul 09 14:11:20.708841 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:11:20.708822    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4\": container with ID starting with b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4 not found: ID does not exist" containerID="b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"
Jul 09 14:11:20.708887 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.708848    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4"} err="failed to get container status \"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4\": rpc error: code = NotFound desc = could not find container \"b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4\": container with ID starting with b68f923b23cea0afa47818c8df53855dee4e7f7ab5e1e745c7d55f1f69c163b4 not found: ID does not exist"
Jul 09 14:11:20.722399 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.722373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:11:20.726621 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:20.726597    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-vlzws"]
Jul 09 14:11:21.060843 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.060809    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" path="/var/lib/kubelet/pods/eac44ee9-7b68-4e39-a23b-71e78c2a085c/volumes"
Jul 09 14:11:21.502013 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.501930    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-gcn8x"]
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502275    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2b6aea47-78f5-4761-a241-9b55cfc6b635" containerName="authorino"
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502286    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2b6aea47-78f5-4761-a241-9b55cfc6b635" containerName="authorino"
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502303    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" containerName="manager"
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502308    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" containerName="manager"
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502368    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eac44ee9-7b68-4e39-a23b-71e78c2a085c" containerName="manager"
Jul 09 14:11:21.502403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.502377    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2b6aea47-78f5-4761-a241-9b55cfc6b635" containerName="authorino"
Jul 09 14:11:21.506632 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.506613    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.509726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.509696    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ln8rm\""
Jul 09 14:11:21.509726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.509698    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 14:11:21.509908 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.509738    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 14:11:21.512846 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.512820    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-gcn8x"]
Jul 09 14:11:21.648679 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.648636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f55597ad-2cad-4be9-bb96-81d28e58ef95-webhook-cert\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.648878 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.648781    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2fg\" (UniqueName: \"kubernetes.io/projected/f55597ad-2cad-4be9-bb96-81d28e58ef95-kube-api-access-9z2fg\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.749628 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.749588    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f55597ad-2cad-4be9-bb96-81d28e58ef95-webhook-cert\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.749839 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.749688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2fg\" (UniqueName: \"kubernetes.io/projected/f55597ad-2cad-4be9-bb96-81d28e58ef95-kube-api-access-9z2fg\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.752352 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.752289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f55597ad-2cad-4be9-bb96-81d28e58ef95-webhook-cert\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.761525 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.761497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2fg\" (UniqueName: \"kubernetes.io/projected/f55597ad-2cad-4be9-bb96-81d28e58ef95-kube-api-access-9z2fg\") pod \"maas-controller-849cbdf4c4-gcn8x\" (UID: \"f55597ad-2cad-4be9-bb96-81d28e58ef95\") " pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.818490 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.818432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:21.964445 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:21.964406    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-849cbdf4c4-gcn8x"]
Jul 09 14:11:21.966781 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:11:21.966746    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf55597ad_2cad_4be9_bb96_81d28e58ef95.slice/crio-968c36c67512f5947b6b3c13b5d8ebe59ba770f0e492d867a7b9657578fbba5f WatchSource:0}: Error finding container 968c36c67512f5947b6b3c13b5d8ebe59ba770f0e492d867a7b9657578fbba5f: Status 404 returned error can't find the container with id 968c36c67512f5947b6b3c13b5d8ebe59ba770f0e492d867a7b9657578fbba5f
Jul 09 14:11:22.712189 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:22.712149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x" event={"ID":"f55597ad-2cad-4be9-bb96-81d28e58ef95","Type":"ContainerStarted","Data":"9b4c7af68623f486134ed8bfc381032dc7f0ed668008b04243dd0ecd1c2b6ded"}
Jul 09 14:11:22.712643 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:22.712217    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:11:22.712643 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:22.712234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x" event={"ID":"f55597ad-2cad-4be9-bb96-81d28e58ef95","Type":"ContainerStarted","Data":"968c36c67512f5947b6b3c13b5d8ebe59ba770f0e492d867a7b9657578fbba5f"}
Jul 09 14:11:22.728615 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:22.728537    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x" podStartSLOduration=1.096460098 podStartE2EDuration="1.728514691s" totalImagesPullingTime="632.054593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:11:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:11:22.727785082 +0000 UTC m=+788.269051869" watchObservedRunningTime="2026-07-09 14:11:22.728514691 +0000 UTC m=+788.269781474"
Jul 09 14:11:33.723132 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:11:33.723101    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-849cbdf4c4-gcn8x"
Jul 09 14:13:14.943986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:13:14.943895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:13:14.948548 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:13:14.948523    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:13:14.950730 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:13:14.950710    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:13:14.957620 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:13:14.957598    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:18:14.981257 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:18:14.981221    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:18:14.982166 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:18:14.982133    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:18:14.987692 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:18:14.987669    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:18:14.988331 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:18:14.988312    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:23:15.011880 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.011846    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:23:15.012836 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.012811    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:23:15.020121 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.020099    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:23:15.021917 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.021891    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:23:15.815920 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.815887    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:15.819434 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.819394    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.822215 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.822191    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c6ebe748-serving-cert\""
Jul 09 14:23:15.846967 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.846932    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:15.890899 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.890857    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.891132 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.891000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.891132 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.891049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djs2m\" (UniqueName: \"kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.992137 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.992089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.992349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.992177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.992349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.992202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djs2m\" (UniqueName: \"kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:15.992349 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:15.992244    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-serving-cert: secret "maas-api-e2e-ait-c6ebe748-serving-cert" not found
Jul 09 14:23:15.992349 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:15.992335    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls podName:b4f1aff6-a36c-4c86-80a0-e434d165e28b nodeName:}" failed. No retries permitted until 2026-07-09 14:23:16.49230945 +0000 UTC m=+1502.033576212 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls") pod "maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" (UID: "b4f1aff6-a36c-4c86-80a0-e434d165e28b") : secret "maas-api-e2e-ait-c6ebe748-serving-cert" not found
Jul 09 14:23:15.992952 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:15.992925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:16.002686 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.002659    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djs2m\" (UniqueName: \"kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:16.497984 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.497950    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:16.500629 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.500602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") pod \"maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:16.730187 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.730154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:16.881847 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.881818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:16.883945 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:23:16.883915    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4f1aff6_a36c_4c86_80a0_e434d165e28b.slice/crio-f481ec0c8abd6cb5bb3d76ef843286a443916e5acce8e759cc4389056fd68a66 WatchSource:0}: Error finding container f481ec0c8abd6cb5bb3d76ef843286a443916e5acce8e759cc4389056fd68a66: Status 404 returned error can't find the container with id f481ec0c8abd6cb5bb3d76ef843286a443916e5acce8e759cc4389056fd68a66
Jul 09 14:23:16.885805 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:16.885783    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:23:17.051705 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:17.051611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" event={"ID":"b4f1aff6-a36c-4c86-80a0-e434d165e28b","Type":"ContainerStarted","Data":"f481ec0c8abd6cb5bb3d76ef843286a443916e5acce8e759cc4389056fd68a66"}
Jul 09 14:23:18.057973 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:18.057934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" event={"ID":"b4f1aff6-a36c-4c86-80a0-e434d165e28b","Type":"ContainerStarted","Data":"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"}
Jul 09 14:23:18.058473 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:18.058045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:18.076888 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:18.076820    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" podStartSLOduration=2.380489277 podStartE2EDuration="3.07679688s" totalImagesPullingTime="696.307603ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:23:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:23:18.075115747 +0000 UTC m=+1503.616382550" watchObservedRunningTime="2026-07-09 14:23:18.07679688 +0000 UTC m=+1503.618063663"
Jul 09 14:23:21.146960 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.146917    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:21.147433 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.147241    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" podUID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" containerName="maas-api" containerID="cri-o://11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157" gracePeriod=30
Jul 09 14:23:21.403253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.403188    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:21.545621 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.545559    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djs2m\" (UniqueName: \"kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m\") pod \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") "
Jul 09 14:23:21.545806 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.545671    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca\") pod \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") "
Jul 09 14:23:21.545806 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.545750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") pod \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\" (UID: \"b4f1aff6-a36c-4c86-80a0-e434d165e28b\") "
Jul 09 14:23:21.546043 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.546014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4f1aff6-a36c-4c86-80a0-e434d165e28b" (UID: "b4f1aff6-a36c-4c86-80a0-e434d165e28b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:23:21.548534 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.548495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4f1aff6-a36c-4c86-80a0-e434d165e28b" (UID: "b4f1aff6-a36c-4c86-80a0-e434d165e28b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:23:21.548955 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.548933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m" (OuterVolumeSpecName: "kube-api-access-djs2m") pod "b4f1aff6-a36c-4c86-80a0-e434d165e28b" (UID: "b4f1aff6-a36c-4c86-80a0-e434d165e28b"). InnerVolumeSpecName "kube-api-access-djs2m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:23:21.646820 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.646774    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4f1aff6-a36c-4c86-80a0-e434d165e28b-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:21.646820 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.646812    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4f1aff6-a36c-4c86-80a0-e434d165e28b-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:21.646820 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:21.646821    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djs2m\" (UniqueName: \"kubernetes.io/projected/b4f1aff6-a36c-4c86-80a0-e434d165e28b-kube-api-access-djs2m\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:22.079269 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.079230    2580 generic.go:356] "Generic (PLEG): container finished" podID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" containerID="11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157" exitCode=0
Jul 09 14:23:22.079467 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.079277    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"
Jul 09 14:23:22.079467 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.079318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" event={"ID":"b4f1aff6-a36c-4c86-80a0-e434d165e28b","Type":"ContainerDied","Data":"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"}
Jul 09 14:23:22.079467 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.079372    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb" event={"ID":"b4f1aff6-a36c-4c86-80a0-e434d165e28b","Type":"ContainerDied","Data":"f481ec0c8abd6cb5bb3d76ef843286a443916e5acce8e759cc4389056fd68a66"}
Jul 09 14:23:22.079467 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.079404    2580 scope.go:117] "RemoveContainer" containerID="11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"
Jul 09 14:23:22.089061 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.089041    2580 scope.go:117] "RemoveContainer" containerID="11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"
Jul 09 14:23:22.089357 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:22.089340    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157\": container with ID starting with 11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157 not found: ID does not exist" containerID="11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"
Jul 09 14:23:22.089402 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.089367    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157"} err="failed to get container status \"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157\": rpc error: code = NotFound desc = could not find container \"11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157\": container with ID starting with 11018139d104d4c3a46b4eca5dde0c7fad2e75f378b1af239b09777e24811157 not found: ID does not exist"
Jul 09 14:23:22.100582 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.100544    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:22.104008 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:22.103984    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c6ebe748-58c4f745bd-62msb"]
Jul 09 14:23:23.060900 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:23.060865    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" path="/var/lib/kubelet/pods/b4f1aff6-a36c-4c86-80a0-e434d165e28b/volumes"
Jul 09 14:23:28.685863 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.685821    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:28.686377 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.686362    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" containerName="maas-api"
Jul 09 14:23:28.686421 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.686381    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" containerName="maas-api"
Jul 09 14:23:28.686500 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.686489    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4f1aff6-a36c-4c86-80a0-e434d165e28b" containerName="maas-api"
Jul 09 14:23:28.691363 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.691331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.693968 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.693936    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-f7f85c0a-serving-cert\""
Jul 09 14:23:28.700659 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.700274    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:28.813332 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.813283    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.813332 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.813328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.813615 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.813357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8vjj\" (UniqueName: \"kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.914115 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.914073    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.914115 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.914121    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.914353 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.914150    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8vjj\" (UniqueName: \"kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.914952 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.914866    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.916999 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.916964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:28.922580 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:28.922526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8vjj\" (UniqueName: \"kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj\") pod \"maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:29.005758 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:29.005664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:29.156371 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:29.156272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:29.159607 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:23:29.159553    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9c83278_778d_4b77_a830_360727df4d73.slice/crio-42c64ac59483cc072ad6e0c2186f3c897bd69f412a3567e8bb46e081651b2e20 WatchSource:0}: Error finding container 42c64ac59483cc072ad6e0c2186f3c897bd69f412a3567e8bb46e081651b2e20: Status 404 returned error can't find the container with id 42c64ac59483cc072ad6e0c2186f3c897bd69f412a3567e8bb46e081651b2e20
Jul 09 14:23:30.120713 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:30.120669    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" event={"ID":"a9c83278-778d-4b77-a830-360727df4d73","Type":"ContainerStarted","Data":"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"}
Jul 09 14:23:30.121148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:30.120732    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" event={"ID":"a9c83278-778d-4b77-a830-360727df4d73","Type":"ContainerStarted","Data":"42c64ac59483cc072ad6e0c2186f3c897bd69f412a3567e8bb46e081651b2e20"}
Jul 09 14:23:30.121148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:30.120754    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:30.143171 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:30.143122    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" podStartSLOduration=1.61071024 podStartE2EDuration="2.14310496s" totalImagesPullingTime="532.39472ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:23:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:23:30.14085515 +0000 UTC m=+1515.682121955" watchObservedRunningTime="2026-07-09 14:23:30.14310496 +0000 UTC m=+1515.684371742"
Jul 09 14:23:33.649224 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.649161    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:33.650242 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.650187    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" podUID="a9c83278-778d-4b77-a830-360727df4d73" containerName="maas-api" containerID="cri-o://c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b" gracePeriod=30
Jul 09 14:23:33.910680 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.910616    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:33.966993 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.966960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls\") pod \"a9c83278-778d-4b77-a830-360727df4d73\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") "
Jul 09 14:23:33.967201 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.967028    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8vjj\" (UniqueName: \"kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj\") pod \"a9c83278-778d-4b77-a830-360727df4d73\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") "
Jul 09 14:23:33.967201 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.967060    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca\") pod \"a9c83278-778d-4b77-a830-360727df4d73\" (UID: \"a9c83278-778d-4b77-a830-360727df4d73\") "
Jul 09 14:23:33.967361 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.967335    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9c83278-778d-4b77-a830-360727df4d73" (UID: "a9c83278-778d-4b77-a830-360727df4d73"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:23:33.969307 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.969277    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9c83278-778d-4b77-a830-360727df4d73" (UID: "a9c83278-778d-4b77-a830-360727df4d73"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:23:33.969411 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:33.969330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj" (OuterVolumeSpecName: "kube-api-access-t8vjj") pod "a9c83278-778d-4b77-a830-360727df4d73" (UID: "a9c83278-778d-4b77-a830-360727df4d73"). InnerVolumeSpecName "kube-api-access-t8vjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:23:34.068003 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.067949    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9c83278-778d-4b77-a830-360727df4d73-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:34.068003 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.068000    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9c83278-778d-4b77-a830-360727df4d73-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:34.068003 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.068011    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8vjj\" (UniqueName: \"kubernetes.io/projected/a9c83278-778d-4b77-a830-360727df4d73-kube-api-access-t8vjj\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:34.141283 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.141246    2580 generic.go:356] "Generic (PLEG): container finished" podID="a9c83278-778d-4b77-a830-360727df4d73" containerID="c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b" exitCode=0
Jul 09 14:23:34.141458 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.141299    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"
Jul 09 14:23:34.141458 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.141329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" event={"ID":"a9c83278-778d-4b77-a830-360727df4d73","Type":"ContainerDied","Data":"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"}
Jul 09 14:23:34.141458 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.141385    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f" event={"ID":"a9c83278-778d-4b77-a830-360727df4d73","Type":"ContainerDied","Data":"42c64ac59483cc072ad6e0c2186f3c897bd69f412a3567e8bb46e081651b2e20"}
Jul 09 14:23:34.141458 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.141407    2580 scope.go:117] "RemoveContainer" containerID="c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"
Jul 09 14:23:34.150773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.150755    2580 scope.go:117] "RemoveContainer" containerID="c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"
Jul 09 14:23:34.151060 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:34.151038    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b\": container with ID starting with c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b not found: ID does not exist" containerID="c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"
Jul 09 14:23:34.151135 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.151071    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b"} err="failed to get container status \"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b\": rpc error: code = NotFound desc = could not find container \"c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b\": container with ID starting with c734c4c8ea9e0112f2c55c46c722b6e74ea58c5b189e3d03e43cbd03a3336b9b not found: ID does not exist"
Jul 09 14:23:34.162608 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.162493    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:34.166746 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:34.166707    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-f7f85c0a-6bc5cd9cdf-2ph9f"]
Jul 09 14:23:35.060534 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:35.060498    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9c83278-778d-4b77-a830-360727df4d73" path="/var/lib/kubelet/pods/a9c83278-778d-4b77-a830-360727df4d73/volumes"
Jul 09 14:23:40.813767 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.813723    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:40.814349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.814332    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9c83278-778d-4b77-a830-360727df4d73" containerName="maas-api"
Jul 09 14:23:40.814387 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.814353    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9c83278-778d-4b77-a830-360727df4d73" containerName="maas-api"
Jul 09 14:23:40.814444 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.814433    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9c83278-778d-4b77-a830-360727df4d73" containerName="maas-api"
Jul 09 14:23:40.819325 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.819303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:40.823181 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.823156    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0fc73082-serving-cert\""
Jul 09 14:23:40.835115 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.835083    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:40.930689 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.930650    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:40.930891 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.930715    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stbcv\" (UniqueName: \"kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:40.930891 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:40.930744    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.031999 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.031947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stbcv\" (UniqueName: \"kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.031999 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.032004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.032284 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.032091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.032284 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:41.032173    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-serving-cert: secret "maas-api-e2e-ait-0fc73082-serving-cert" not found
Jul 09 14:23:41.032284 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:41.032239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls podName:3ffb6652-a3ad-4658-bfea-dd3af6780485 nodeName:}" failed. No retries permitted until 2026-07-09 14:23:41.532222822 +0000 UTC m=+1527.073489583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls") pod "maas-api-e2e-ait-0fc73082-5475489-cm4z7" (UID: "3ffb6652-a3ad-4658-bfea-dd3af6780485") : secret "maas-api-e2e-ait-0fc73082-serving-cert" not found
Jul 09 14:23:41.032699 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.032681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.045119 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.045091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stbcv\" (UniqueName: \"kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.538415 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.538368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.541150 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.541128    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") pod \"maas-api-e2e-ait-0fc73082-5475489-cm4z7\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.730348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.730305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:41.873879 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:41.873749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:41.876845 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:23:41.876808    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ffb6652_a3ad_4658_bfea_dd3af6780485.slice/crio-155eec2aa1cc113c56d2493c9335fd443ef1578012083608d5dbd12d7ea69829 WatchSource:0}: Error finding container 155eec2aa1cc113c56d2493c9335fd443ef1578012083608d5dbd12d7ea69829: Status 404 returned error can't find the container with id 155eec2aa1cc113c56d2493c9335fd443ef1578012083608d5dbd12d7ea69829
Jul 09 14:23:42.180430 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:42.180327    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" event={"ID":"3ffb6652-a3ad-4658-bfea-dd3af6780485","Type":"ContainerStarted","Data":"155eec2aa1cc113c56d2493c9335fd443ef1578012083608d5dbd12d7ea69829"}
Jul 09 14:23:43.187747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:43.187708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" event={"ID":"3ffb6652-a3ad-4658-bfea-dd3af6780485","Type":"ContainerStarted","Data":"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"}
Jul 09 14:23:43.188119 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:43.187802    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:43.234367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:43.234304    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" podStartSLOduration=2.353163242 podStartE2EDuration="3.234279498s" totalImagesPullingTime="881.116256ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:23:43.231069151 +0000 UTC m=+1528.772335935" watchObservedRunningTime="2026-07-09 14:23:43.234279498 +0000 UTC m=+1528.775546282"
Jul 09 14:23:46.344892 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.344849    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:46.345459 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.345087    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" podUID="3ffb6652-a3ad-4658-bfea-dd3af6780485" containerName="maas-api" containerID="cri-o://aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e" gracePeriod=30
Jul 09 14:23:46.597919 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.597855    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:46.688781 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.688742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca\") pod \"3ffb6652-a3ad-4658-bfea-dd3af6780485\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") "
Jul 09 14:23:46.689064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.688820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-stbcv\" (UniqueName: \"kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv\") pod \"3ffb6652-a3ad-4658-bfea-dd3af6780485\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") "
Jul 09 14:23:46.689064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.688862    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") pod \"3ffb6652-a3ad-4658-bfea-dd3af6780485\" (UID: \"3ffb6652-a3ad-4658-bfea-dd3af6780485\") "
Jul 09 14:23:46.689152 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.689087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3ffb6652-a3ad-4658-bfea-dd3af6780485" (UID: "3ffb6652-a3ad-4658-bfea-dd3af6780485"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:23:46.691206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.691177    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3ffb6652-a3ad-4658-bfea-dd3af6780485" (UID: "3ffb6652-a3ad-4658-bfea-dd3af6780485"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:23:46.691206 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.691192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv" (OuterVolumeSpecName: "kube-api-access-stbcv") pod "3ffb6652-a3ad-4658-bfea-dd3af6780485" (UID: "3ffb6652-a3ad-4658-bfea-dd3af6780485"). InnerVolumeSpecName "kube-api-access-stbcv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:23:46.790182 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.790146    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-stbcv\" (UniqueName: \"kubernetes.io/projected/3ffb6652-a3ad-4658-bfea-dd3af6780485-kube-api-access-stbcv\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:46.790182 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.790179    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3ffb6652-a3ad-4658-bfea-dd3af6780485-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:46.790182 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:46.790191    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ffb6652-a3ad-4658-bfea-dd3af6780485-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:23:47.209043 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.209006    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ffb6652-a3ad-4658-bfea-dd3af6780485" containerID="aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e" exitCode=0
Jul 09 14:23:47.209229 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.209058    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"
Jul 09 14:23:47.209229 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.209048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" event={"ID":"3ffb6652-a3ad-4658-bfea-dd3af6780485","Type":"ContainerDied","Data":"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"}
Jul 09 14:23:47.209229 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.209196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7" event={"ID":"3ffb6652-a3ad-4658-bfea-dd3af6780485","Type":"ContainerDied","Data":"155eec2aa1cc113c56d2493c9335fd443ef1578012083608d5dbd12d7ea69829"}
Jul 09 14:23:47.209386 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.209233    2580 scope.go:117] "RemoveContainer" containerID="aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"
Jul 09 14:23:47.219286 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.219268    2580 scope.go:117] "RemoveContainer" containerID="aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"
Jul 09 14:23:47.219561 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:47.219541    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e\": container with ID starting with aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e not found: ID does not exist" containerID="aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"
Jul 09 14:23:47.219637 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.219588    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e"} err="failed to get container status \"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e\": rpc error: code = NotFound desc = could not find container \"aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e\": container with ID starting with aac51b89ef2a253370d2bf843200c62088a8c7921cea684eb69eeeaa484ef40e not found: ID does not exist"
Jul 09 14:23:47.226923 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.226902    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:47.231761 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:47.231739    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0fc73082-5475489-cm4z7"]
Jul 09 14:23:49.060985 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:49.060949    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ffb6652-a3ad-4658-bfea-dd3af6780485" path="/var/lib/kubelet/pods/3ffb6652-a3ad-4658-bfea-dd3af6780485/volumes"
Jul 09 14:23:59.571295 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.571259    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:23:59.571725 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.571663    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ffb6652-a3ad-4658-bfea-dd3af6780485" containerName="maas-api"
Jul 09 14:23:59.571725 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.571675    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ffb6652-a3ad-4658-bfea-dd3af6780485" containerName="maas-api"
Jul 09 14:23:59.571828 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.571739    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ffb6652-a3ad-4658-bfea-dd3af6780485" containerName="maas-api"
Jul 09 14:23:59.576304 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.576276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.579226 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.579205    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-387a3dbb-serving-cert\""
Jul 09 14:23:59.596031 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.596003    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:23:59.702770 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.702738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.702964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.702803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.702964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.702831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm99q\" (UniqueName: \"kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.804000 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.803957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.804191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.804027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.804191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.804059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm99q\" (UniqueName: \"kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.804191 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:59.804119    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-serving-cert: secret "maas-api-e2e-derive-387a3dbb-serving-cert" not found
Jul 09 14:23:59.804314 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:23:59.804208    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls podName:ea3e2dab-3067-4e6a-874b-2fb96c229016 nodeName:}" failed. No retries permitted until 2026-07-09 14:24:00.304187998 +0000 UTC m=+1545.845454760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls") pod "maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" (UID: "ea3e2dab-3067-4e6a-874b-2fb96c229016") : secret "maas-api-e2e-derive-387a3dbb-serving-cert" not found
Jul 09 14:23:59.804845 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.804826    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:23:59.822542 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:23:59.822463    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm99q\" (UniqueName: \"kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:00.308918 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:00.308889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:00.311616 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:00.311591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") pod \"maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:00.486603 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:00.486549    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:00.631110 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:00.631081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:24:00.633517 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:24:00.633473    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3e2dab_3067_4e6a_874b_2fb96c229016.slice/crio-2035f41304941edc3cf5d0a3e87ead1f9f131b358c4ce040ac1a378c10be25d8 WatchSource:0}: Error finding container 2035f41304941edc3cf5d0a3e87ead1f9f131b358c4ce040ac1a378c10be25d8: Status 404 returned error can't find the container with id 2035f41304941edc3cf5d0a3e87ead1f9f131b358c4ce040ac1a378c10be25d8
Jul 09 14:24:01.277755 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:01.277720    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" event={"ID":"ea3e2dab-3067-4e6a-874b-2fb96c229016","Type":"ContainerStarted","Data":"095097ae31d28296acded204c4b353e56e80395b0cdf416629f3f34f37c1a63c"}
Jul 09 14:24:01.277965 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:01.277774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" event={"ID":"ea3e2dab-3067-4e6a-874b-2fb96c229016","Type":"ContainerStarted","Data":"2035f41304941edc3cf5d0a3e87ead1f9f131b358c4ce040ac1a378c10be25d8"}
Jul 09 14:24:01.277965 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:01.277825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:01.295944 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:01.295897    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" podStartSLOduration=1.860464519 podStartE2EDuration="2.295878928s" totalImagesPullingTime="435.414409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:23:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:24:01.294723385 +0000 UTC m=+1546.835990183" watchObservedRunningTime="2026-07-09 14:24:01.295878928 +0000 UTC m=+1546.837145710"
Jul 09 14:24:04.154928 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.154887    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:24:04.155533 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.155282    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" podUID="ea3e2dab-3067-4e6a-874b-2fb96c229016" containerName="maas-api" containerID="cri-o://095097ae31d28296acded204c4b353e56e80395b0cdf416629f3f34f37c1a63c" gracePeriod=30
Jul 09 14:24:04.300172 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.300141    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea3e2dab-3067-4e6a-874b-2fb96c229016" containerID="095097ae31d28296acded204c4b353e56e80395b0cdf416629f3f34f37c1a63c" exitCode=0
Jul 09 14:24:04.300338 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.300204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" event={"ID":"ea3e2dab-3067-4e6a-874b-2fb96c229016","Type":"ContainerDied","Data":"095097ae31d28296acded204c4b353e56e80395b0cdf416629f3f34f37c1a63c"}
Jul 09 14:24:04.420351 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.420274    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:04.551233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.551190    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca\") pod \"ea3e2dab-3067-4e6a-874b-2fb96c229016\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") "
Jul 09 14:24:04.551233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.551233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm99q\" (UniqueName: \"kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q\") pod \"ea3e2dab-3067-4e6a-874b-2fb96c229016\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") "
Jul 09 14:24:04.551487 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.551311    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") pod \"ea3e2dab-3067-4e6a-874b-2fb96c229016\" (UID: \"ea3e2dab-3067-4e6a-874b-2fb96c229016\") "
Jul 09 14:24:04.551550 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.551527    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ea3e2dab-3067-4e6a-874b-2fb96c229016" (UID: "ea3e2dab-3067-4e6a-874b-2fb96c229016"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:24:04.551655 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.551628    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ea3e2dab-3067-4e6a-874b-2fb96c229016-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:24:04.553505 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.553482    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ea3e2dab-3067-4e6a-874b-2fb96c229016" (UID: "ea3e2dab-3067-4e6a-874b-2fb96c229016"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:24:04.553610 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.553534    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q" (OuterVolumeSpecName: "kube-api-access-fm99q") pod "ea3e2dab-3067-4e6a-874b-2fb96c229016" (UID: "ea3e2dab-3067-4e6a-874b-2fb96c229016"). InnerVolumeSpecName "kube-api-access-fm99q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:24:04.652397 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.652358    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ea3e2dab-3067-4e6a-874b-2fb96c229016-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:24:04.652397 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:04.652391    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm99q\" (UniqueName: \"kubernetes.io/projected/ea3e2dab-3067-4e6a-874b-2fb96c229016-kube-api-access-fm99q\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:24:05.306457 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:05.306425    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"
Jul 09 14:24:05.306938 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:05.306470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv" event={"ID":"ea3e2dab-3067-4e6a-874b-2fb96c229016","Type":"ContainerDied","Data":"2035f41304941edc3cf5d0a3e87ead1f9f131b358c4ce040ac1a378c10be25d8"}
Jul 09 14:24:05.306938 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:05.306531    2580 scope.go:117] "RemoveContainer" containerID="095097ae31d28296acded204c4b353e56e80395b0cdf416629f3f34f37c1a63c"
Jul 09 14:24:05.322596 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:05.322543    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:24:05.328123 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:05.328097    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-387a3dbb-85684f7fd5-jggnv"]
Jul 09 14:24:07.060768 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:24:07.060736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea3e2dab-3067-4e6a-874b-2fb96c229016" path="/var/lib/kubelet/pods/ea3e2dab-3067-4e6a-874b-2fb96c229016/volumes"
Jul 09 14:25:48.440660 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.440622    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:25:48.441214 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.441192    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea3e2dab-3067-4e6a-874b-2fb96c229016" containerName="maas-api"
Jul 09 14:25:48.441295 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.441218    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea3e2dab-3067-4e6a-874b-2fb96c229016" containerName="maas-api"
Jul 09 14:25:48.441347 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.441311    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea3e2dab-3067-4e6a-874b-2fb96c229016" containerName="maas-api"
Jul 09 14:25:48.444606 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.444581    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.447170 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.447147    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-a0eef674-serving-cert\""
Jul 09 14:25:48.455198 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.455170    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:25:48.497971 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.497933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khjgj\" (UniqueName: \"kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.498180 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.497981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.498180 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.498009    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.598771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.598725    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.598972 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.598837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khjgj\" (UniqueName: \"kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.598972 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.598865    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.599111 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:25:48.598975    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-serving-cert: secret "maas-api-e2e-mt-a0eef674-serving-cert" not found
Jul 09 14:25:48.599111 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:25:48.599046    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls podName:4b290572-e8bb-4e3d-b24a-4980eb956889 nodeName:}" failed. No retries permitted until 2026-07-09 14:25:49.099024153 +0000 UTC m=+1654.640290914 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls") pod "maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" (UID: "4b290572-e8bb-4e3d-b24a-4980eb956889") : secret "maas-api-e2e-mt-a0eef674-serving-cert" not found
Jul 09 14:25:48.599509 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.599483    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:48.607656 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:48.607621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khjgj\" (UniqueName: \"kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:49.103698 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:49.103656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:49.106262 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:49.106240    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") pod \"maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:49.356504 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:49.356405    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:49.511386 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:49.511358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:25:49.514269 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:25:49.514237    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b290572_e8bb_4e3d_b24a_4980eb956889.slice/crio-ce99c1fb4312301a2417d3ec675164cba2fabf392493295e0dddd1d3122c2e84 WatchSource:0}: Error finding container ce99c1fb4312301a2417d3ec675164cba2fabf392493295e0dddd1d3122c2e84: Status 404 returned error can't find the container with id ce99c1fb4312301a2417d3ec675164cba2fabf392493295e0dddd1d3122c2e84
Jul 09 14:25:49.800793 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:49.800693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" event={"ID":"4b290572-e8bb-4e3d-b24a-4980eb956889","Type":"ContainerStarted","Data":"ce99c1fb4312301a2417d3ec675164cba2fabf392493295e0dddd1d3122c2e84"}
Jul 09 14:25:50.807279 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:50.807240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" event={"ID":"4b290572-e8bb-4e3d-b24a-4980eb956889","Type":"ContainerStarted","Data":"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"}
Jul 09 14:25:50.807712 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:50.807502    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:25:50.826000 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:50.825951    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" podStartSLOduration=2.341373056 podStartE2EDuration="2.825936991s" totalImagesPullingTime="484.563935ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:25:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:25:50.82453676 +0000 UTC m=+1656.365803565" watchObservedRunningTime="2026-07-09 14:25:50.825936991 +0000 UTC m=+1656.367203774"
Jul 09 14:25:56.817623 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:25:56.817563    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:26:00.426177 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.426143    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:26:00.426602 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.426368    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" podUID="4b290572-e8bb-4e3d-b24a-4980eb956889" containerName="maas-api" containerID="cri-o://e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705" gracePeriod=30
Jul 09 14:26:00.678191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.678122    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:26:00.814923 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.814887    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-khjgj\" (UniqueName: \"kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj\") pod \"4b290572-e8bb-4e3d-b24a-4980eb956889\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") "
Jul 09 14:26:00.815102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.815011    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") pod \"4b290572-e8bb-4e3d-b24a-4980eb956889\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") "
Jul 09 14:26:00.815102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.815049    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca\") pod \"4b290572-e8bb-4e3d-b24a-4980eb956889\" (UID: \"4b290572-e8bb-4e3d-b24a-4980eb956889\") "
Jul 09 14:26:00.815479 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.815446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b290572-e8bb-4e3d-b24a-4980eb956889" (UID: "4b290572-e8bb-4e3d-b24a-4980eb956889"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:26:00.817231 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.817209    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj" (OuterVolumeSpecName: "kube-api-access-khjgj") pod "4b290572-e8bb-4e3d-b24a-4980eb956889" (UID: "4b290572-e8bb-4e3d-b24a-4980eb956889"). InnerVolumeSpecName "kube-api-access-khjgj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:26:00.817338 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.817267    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b290572-e8bb-4e3d-b24a-4980eb956889" (UID: "4b290572-e8bb-4e3d-b24a-4980eb956889"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:26:00.855453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.855406    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b290572-e8bb-4e3d-b24a-4980eb956889" containerID="e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705" exitCode=0
Jul 09 14:26:00.855453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.855438    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"
Jul 09 14:26:00.855702 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.855484    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" event={"ID":"4b290572-e8bb-4e3d-b24a-4980eb956889","Type":"ContainerDied","Data":"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"}
Jul 09 14:26:00.855702 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.855536    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4" event={"ID":"4b290572-e8bb-4e3d-b24a-4980eb956889","Type":"ContainerDied","Data":"ce99c1fb4312301a2417d3ec675164cba2fabf392493295e0dddd1d3122c2e84"}
Jul 09 14:26:00.855702 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.855599    2580 scope.go:117] "RemoveContainer" containerID="e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"
Jul 09 14:26:00.864883 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.864866    2580 scope.go:117] "RemoveContainer" containerID="e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"
Jul 09 14:26:00.865147 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:26:00.865126    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705\": container with ID starting with e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705 not found: ID does not exist" containerID="e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"
Jul 09 14:26:00.865199 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.865156    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705"} err="failed to get container status \"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705\": rpc error: code = NotFound desc = could not find container \"e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705\": container with ID starting with e8e8ec66f6c7f6fba38b3e1c29f42b1e29f1e2620f1c0789a0e283dd4cb5b705 not found: ID does not exist"
Jul 09 14:26:00.880064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.880013    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:26:00.882297 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.882272    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-a0eef674-86cc6b859-tv4c4"]
Jul 09 14:26:00.916145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.916089    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b290572-e8bb-4e3d-b24a-4980eb956889-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:26:00.916145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.916135    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-khjgj\" (UniqueName: \"kubernetes.io/projected/4b290572-e8bb-4e3d-b24a-4980eb956889-kube-api-access-khjgj\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:26:00.916145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:00.916146    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b290572-e8bb-4e3d-b24a-4980eb956889-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:26:01.061417 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:01.061385    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b290572-e8bb-4e3d-b24a-4980eb956889" path="/var/lib/kubelet/pods/4b290572-e8bb-4e3d-b24a-4980eb956889/volumes"
Jul 09 14:26:29.699519 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.699471    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:26:29.700194 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.700019    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b290572-e8bb-4e3d-b24a-4980eb956889" containerName="maas-api"
Jul 09 14:26:29.700194 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.700035    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b290572-e8bb-4e3d-b24a-4980eb956889" containerName="maas-api"
Jul 09 14:26:29.700194 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.700146    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b290572-e8bb-4e3d-b24a-4980eb956889" containerName="maas-api"
Jul 09 14:26:29.704786 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.704758    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.707453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.707425    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-546e6b43-serving-cert\""
Jul 09 14:26:29.715322 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.715288    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:26:29.786497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.786454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.786497 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.786507    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.786798 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.786657    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgjfb\" (UniqueName: \"kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.888013 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.887973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgjfb\" (UniqueName: \"kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.888237 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.888040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.888237 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.888085    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.888237 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:26:29.888185    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-serving-cert: secret "maas-api-e2e-mt-546e6b43-serving-cert" not found
Jul 09 14:26:29.888416 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:26:29.888248    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls podName:9aa838a0-68cf-4b86-ba14-25beb0dc0856 nodeName:}" failed. No retries permitted until 2026-07-09 14:26:30.388231548 +0000 UTC m=+1695.929498309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls") pod "maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" (UID: "9aa838a0-68cf-4b86-ba14-25beb0dc0856") : secret "maas-api-e2e-mt-546e6b43-serving-cert" not found
Jul 09 14:26:29.888854 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.888830    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:29.902080 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:29.902051    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgjfb\" (UniqueName: \"kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:30.393582 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:30.393547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:30.393787 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:26:30.393707    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-serving-cert: secret "maas-api-e2e-mt-546e6b43-serving-cert" not found
Jul 09 14:26:30.393840 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:26:30.393810    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls podName:9aa838a0-68cf-4b86-ba14-25beb0dc0856 nodeName:}" failed. No retries permitted until 2026-07-09 14:26:31.393789078 +0000 UTC m=+1696.935055854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls") pod "maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" (UID: "9aa838a0-68cf-4b86-ba14-25beb0dc0856") : secret "maas-api-e2e-mt-546e6b43-serving-cert" not found
Jul 09 14:26:31.402330 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:31.402290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:31.405061 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:31.405027    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:31.518867 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:31.518834    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:31.685344 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:31.685316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:26:31.688240 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:26:31.688206    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa838a0_68cf_4b86_ba14_25beb0dc0856.slice/crio-1aba39e44439ce94afc19d88734ed4c929c9df353fa74ce8819119e82abb4e64 WatchSource:0}: Error finding container 1aba39e44439ce94afc19d88734ed4c929c9df353fa74ce8819119e82abb4e64: Status 404 returned error can't find the container with id 1aba39e44439ce94afc19d88734ed4c929c9df353fa74ce8819119e82abb4e64
Jul 09 14:26:32.004181 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:32.004085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" event={"ID":"9aa838a0-68cf-4b86-ba14-25beb0dc0856","Type":"ContainerStarted","Data":"1aba39e44439ce94afc19d88734ed4c929c9df353fa74ce8819119e82abb4e64"}
Jul 09 14:26:33.010884 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:33.010838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" event={"ID":"9aa838a0-68cf-4b86-ba14-25beb0dc0856","Type":"ContainerStarted","Data":"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"}
Jul 09 14:26:33.011278 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:33.011006    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:33.042490 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:33.042435    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" podStartSLOduration=3.404971113 podStartE2EDuration="4.042418875s" totalImagesPullingTime="637.447762ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:26:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:26:33.041106828 +0000 UTC m=+1698.582373633" watchObservedRunningTime="2026-07-09 14:26:33.042418875 +0000 UTC m=+1698.583685658"
Jul 09 14:26:39.022196 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:39.022099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:26:43.246480 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.246439    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:26:43.249731 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.249709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.253450 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.253417    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-546e6b43\"/\"default-dockercfg-76b82\""
Jul 09 14:26:43.253649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.253418    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-546e6b43\"/\"kube-root-ca.crt\""
Jul 09 14:26:43.253649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.253422    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-546e6b43\"/\"e2e-lifecycle-model-546e6b43-kserve-self-signed-certs\""
Jul 09 14:26:43.253764 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.253682    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-546e6b43\"/\"openshift-service-ca.crt\""
Jul 09 14:26:43.260889 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.260860    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:26:43.312014 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.311972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.312014 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.312014    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.312253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.312050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.312253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.312155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.312253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.312197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.312253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.312217    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sscz\" (UniqueName: \"kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.413718 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.413905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413751    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.413905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.413905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sscz\" (UniqueName: \"kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.413905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.414134 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.413924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.414193 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.414141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.414257 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.414216    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.414386 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.414363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.416280 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.416249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.416415 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.416302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.422232 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.422202    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sscz\" (UniqueName: \"kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz\") pod \"e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") " pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.560039 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.560002    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:43.712425 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:43.712388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:26:43.715664 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:26:43.715625    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e4cbd20_a1dd_456e_aa9f_636610aa717d.slice/crio-a7a919bb518784d1fb65312f540f19902c2a19422fac998b9cae7a9454639d34 WatchSource:0}: Error finding container a7a919bb518784d1fb65312f540f19902c2a19422fac998b9cae7a9454639d34: Status 404 returned error can't find the container with id a7a919bb518784d1fb65312f540f19902c2a19422fac998b9cae7a9454639d34
Jul 09 14:26:44.065447 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:44.065406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" event={"ID":"2e4cbd20-a1dd-456e-aa9f-636610aa717d","Type":"ContainerStarted","Data":"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"}
Jul 09 14:26:44.065447 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:44.065453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" event={"ID":"2e4cbd20-a1dd-456e-aa9f-636610aa717d","Type":"ContainerStarted","Data":"a7a919bb518784d1fb65312f540f19902c2a19422fac998b9cae7a9454639d34"}
Jul 09 14:26:44.082380 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:44.082324    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" podStartSLOduration=1.082306209 podStartE2EDuration="1.082306209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:26:44.081495759 +0000 UTC m=+1709.622762555" watchObservedRunningTime="2026-07-09 14:26:44.082306209 +0000 UTC m=+1709.623572992"
Jul 09 14:26:53.560349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:53.560281    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:53.560349 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:53.560355    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:53.573243 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:53.573214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:26:54.121513 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:26:54.121483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:27:07.178441 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.178407    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:27:07.178983 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.178664    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" podUID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" containerName="maas-api" containerID="cri-o://97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23" gracePeriod=30
Jul 09 14:27:07.431714 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.431642    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:27:07.544073 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.544030    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgjfb\" (UniqueName: \"kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb\") pod \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") "
Jul 09 14:27:07.544245 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.544096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca\") pod \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") "
Jul 09 14:27:07.544245 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.544141    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") pod \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\" (UID: \"9aa838a0-68cf-4b86-ba14-25beb0dc0856\") "
Jul 09 14:27:07.544472 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.544446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9aa838a0-68cf-4b86-ba14-25beb0dc0856" (UID: "9aa838a0-68cf-4b86-ba14-25beb0dc0856"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:27:07.546369 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.546338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9aa838a0-68cf-4b86-ba14-25beb0dc0856" (UID: "9aa838a0-68cf-4b86-ba14-25beb0dc0856"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:27:07.546498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.546392    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb" (OuterVolumeSpecName: "kube-api-access-rgjfb") pod "9aa838a0-68cf-4b86-ba14-25beb0dc0856" (UID: "9aa838a0-68cf-4b86-ba14-25beb0dc0856"). InnerVolumeSpecName "kube-api-access-rgjfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:27:07.645348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.645307    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgjfb\" (UniqueName: \"kubernetes.io/projected/9aa838a0-68cf-4b86-ba14-25beb0dc0856-kube-api-access-rgjfb\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:07.645348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.645341    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9aa838a0-68cf-4b86-ba14-25beb0dc0856-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:07.645348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:07.645354    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9aa838a0-68cf-4b86-ba14-25beb0dc0856-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:08.176610 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.176555    2580 generic.go:356] "Generic (PLEG): container finished" podID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" containerID="97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23" exitCode=0
Jul 09 14:27:08.176894 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.176603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" event={"ID":"9aa838a0-68cf-4b86-ba14-25beb0dc0856","Type":"ContainerDied","Data":"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"}
Jul 09 14:27:08.176894 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.176627    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"
Jul 09 14:27:08.176894 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.176679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt" event={"ID":"9aa838a0-68cf-4b86-ba14-25beb0dc0856","Type":"ContainerDied","Data":"1aba39e44439ce94afc19d88734ed4c929c9df353fa74ce8819119e82abb4e64"}
Jul 09 14:27:08.176894 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.176710    2580 scope.go:117] "RemoveContainer" containerID="97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"
Jul 09 14:27:08.186429 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.186273    2580 scope.go:117] "RemoveContainer" containerID="97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"
Jul 09 14:27:08.186712 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:27:08.186665    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23\": container with ID starting with 97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23 not found: ID does not exist" containerID="97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"
Jul 09 14:27:08.186712 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.186696    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23"} err="failed to get container status \"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23\": rpc error: code = NotFound desc = could not find container \"97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23\": container with ID starting with 97873f7246136b9b7c6eeea37684ac5e14112c0b0eab068c58541e819d65eb23 not found: ID does not exist"
Jul 09 14:27:08.197227 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.197197    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:27:08.201464 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:08.201428    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-546e6b43-6df855b6b8-7c6jt"]
Jul 09 14:27:09.061104 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:09.061066    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" path="/var/lib/kubelet/pods/9aa838a0-68cf-4b86-ba14-25beb0dc0856/volumes"
Jul 09 14:27:17.901472 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:17.901432    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:27:32.922322 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:32.922252    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" podUID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" containerName="main" containerID="cri-o://b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930" gracePeriod=45
Jul 09 14:27:33.189139 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.189109    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:27:33.281251 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281209    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sscz\" (UniqueName: \"kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281289    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281731 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281642    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir\") pod \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\" (UID: \"2e4cbd20-a1dd-456e-aa9f-636610aa717d\") "
Jul 09 14:27:33.281731 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281644    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home" (OuterVolumeSpecName: "home") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:27:33.281731 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281723    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:27:33.281952 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281935    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-home\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:33.282028 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281965    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-model-cache\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:33.282028 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.281927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:27:33.283922 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.283888    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm" (OuterVolumeSpecName: "dshm") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:27:33.283922 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.283890    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz" (OuterVolumeSpecName: "kube-api-access-8sscz") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "kube-api-access-8sscz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:27:33.284092 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.283962    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2e4cbd20-a1dd-456e-aa9f-636610aa717d" (UID: "2e4cbd20-a1dd-456e-aa9f-636610aa717d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:27:33.302489 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.302455    2580 generic.go:356] "Generic (PLEG): container finished" podID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" containerID="b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930" exitCode=0
Jul 09 14:27:33.302675 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.302497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" event={"ID":"2e4cbd20-a1dd-456e-aa9f-636610aa717d","Type":"ContainerDied","Data":"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"}
Jul 09 14:27:33.302675 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.302509    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"
Jul 09 14:27:33.302675 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.302547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v" event={"ID":"2e4cbd20-a1dd-456e-aa9f-636610aa717d","Type":"ContainerDied","Data":"a7a919bb518784d1fb65312f540f19902c2a19422fac998b9cae7a9454639d34"}
Jul 09 14:27:33.302675 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.302591    2580 scope.go:117] "RemoveContainer" containerID="b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"
Jul 09 14:27:33.312117 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.312094    2580 scope.go:117] "RemoveContainer" containerID="b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"
Jul 09 14:27:33.312417 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:27:33.312397    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930\": container with ID starting with b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930 not found: ID does not exist" containerID="b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"
Jul 09 14:27:33.312467 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.312428    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930"} err="failed to get container status \"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930\": rpc error: code = NotFound desc = could not find container \"b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930\": container with ID starting with b7f5b78cd923dd73a304c16c3ab59c56d307f5a4c15438f1afd4918e39c5c930 not found: ID does not exist"
Jul 09 14:27:33.329127 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.329088    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:27:33.332370 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.332341    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-546e6b43/e2e-lifecycle-model-546e6b43-kserve-b4f67bbd7-zfc9v"]
Jul 09 14:27:33.383343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.383296    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sscz\" (UniqueName: \"kubernetes.io/projected/2e4cbd20-a1dd-456e-aa9f-636610aa717d-kube-api-access-8sscz\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:33.383343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.383336    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tls-certs\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:33.383343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.383349    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-dshm\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:33.383642 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:33.383357    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e4cbd20-a1dd-456e-aa9f-636610aa717d-tmp-dir\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:27:35.060637 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:27:35.060590    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" path="/var/lib/kubelet/pods/2e4cbd20-a1dd-456e-aa9f-636610aa717d/volumes"
Jul 09 14:28:15.042275 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:28:15.042232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:28:15.046290 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:28:15.046268    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:28:15.049099 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:28:15.049074    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:28:15.052726 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:28:15.052707    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:29:00.466663 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.466603    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467088    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" containerName="main"
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467106    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" containerName="main"
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467138    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" containerName="maas-api"
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467148    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" containerName="maas-api"
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467245    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e4cbd20-a1dd-456e-aa9f-636610aa717d" containerName="main"
Jul 09 14:29:00.467343 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.467259    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aa838a0-68cf-4b86-ba14-25beb0dc0856" containerName="maas-api"
Jul 09 14:29:00.470457 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.470431    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.473228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.473198    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a8a58b-serving-cert\""
Jul 09 14:29:00.480814 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.480788    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:29:00.576752 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.576712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.576752 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.576757    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.577009 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.576782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbgtq\" (UniqueName: \"kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.678035 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.677994    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.678035 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.678043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.678307 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.678161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbgtq\" (UniqueName: \"kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.678834 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.678808    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.680656 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.680639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.687475 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.687453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbgtq\" (UniqueName: \"kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq\") pod \"maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.782864 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.782762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:00.929222 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.929190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:29:00.931648 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:29:00.931619    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85c65c26_fe14_4857_966a_05867187e047.slice/crio-f619ada2b48677a065480426bc3b962bf8c1863871e022216ab81d88d6686542 WatchSource:0}: Error finding container f619ada2b48677a065480426bc3b962bf8c1863871e022216ab81d88d6686542: Status 404 returned error can't find the container with id f619ada2b48677a065480426bc3b962bf8c1863871e022216ab81d88d6686542
Jul 09 14:29:00.933093 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:00.933072    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:29:01.724334 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:01.724295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" event={"ID":"85c65c26-fe14-4857-966a-05867187e047","Type":"ContainerStarted","Data":"f619ada2b48677a065480426bc3b962bf8c1863871e022216ab81d88d6686542"}
Jul 09 14:29:02.731499 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:02.731456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" event={"ID":"85c65c26-fe14-4857-966a-05867187e047","Type":"ContainerStarted","Data":"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"}
Jul 09 14:29:02.731990 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:02.731593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:02.751296 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:02.751243    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" podStartSLOduration=1.670420861 podStartE2EDuration="2.751222496s" totalImagesPullingTime="1.080801635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:02.750333351 +0000 UTC m=+1848.291600157" watchObservedRunningTime="2026-07-09 14:29:02.751222496 +0000 UTC m=+1848.292489279"
Jul 09 14:29:08.660260 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.660224    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:29:08.663973 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.663943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.667002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.666971    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-e75e34-serving-cert\""
Jul 09 14:29:08.682862 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.682824    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:29:08.742308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.742280    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:29:08.750354 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.750320    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.750354 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.750355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.750609 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.750381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6qfc\" (UniqueName: \"kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.851438 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.851398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.851438 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.851434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.851754 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.851451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6qfc\" (UniqueName: \"kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.851754 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:29:08.851602    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-serving-cert: secret "maas-api-e2e-inf-b-e75e34-serving-cert" not found
Jul 09 14:29:08.851754 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:29:08.851679    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls podName:4ef1f501-a9f3-4cc3-8910-d1cb763ffed8 nodeName:}" failed. No retries permitted until 2026-07-09 14:29:09.351655905 +0000 UTC m=+1854.892922681 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls") pod "maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" (UID: "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8") : secret "maas-api-e2e-inf-b-e75e34-serving-cert" not found
Jul 09 14:29:08.852228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.852206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:08.881536 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:08.881505    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6qfc\" (UniqueName: \"kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:09.356501 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:09.356454    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:09.359474 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:09.359440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") pod \"maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:09.578126 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:09.578080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:09.759583 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:09.759536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:29:09.762440 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:29:09.762409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ef1f501_a9f3_4cc3_8910_d1cb763ffed8.slice/crio-4a55252f1f4f09b1d2bc440a47b14706780d959747b8cf24bdc532c4ecda41ab WatchSource:0}: Error finding container 4a55252f1f4f09b1d2bc440a47b14706780d959747b8cf24bdc532c4ecda41ab: Status 404 returned error can't find the container with id 4a55252f1f4f09b1d2bc440a47b14706780d959747b8cf24bdc532c4ecda41ab
Jul 09 14:29:10.772794 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:10.772758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" event={"ID":"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8","Type":"ContainerStarted","Data":"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"}
Jul 09 14:29:10.773263 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:10.772815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" event={"ID":"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8","Type":"ContainerStarted","Data":"4a55252f1f4f09b1d2bc440a47b14706780d959747b8cf24bdc532c4ecda41ab"}
Jul 09 14:29:10.773263 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:10.772870    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:10.793078 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:10.793019    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" podStartSLOduration=2.367513056 podStartE2EDuration="2.792998728s" totalImagesPullingTime="425.485672ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:10.791313618 +0000 UTC m=+1856.332580411" watchObservedRunningTime="2026-07-09 14:29:10.792998728 +0000 UTC m=+1856.334265510"
Jul 09 14:29:16.783413 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:16.783378    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:29:47.353939 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.353895    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:29:47.357276 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.357251    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.360095 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.360073    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e75e34\"/\"test-model-e75e34-kserve-self-signed-certs\""
Jul 09 14:29:47.361053 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.361025    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e75e34\"/\"kube-root-ca.crt\""
Jul 09 14:29:47.361183 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.361057    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-e75e34\"/\"openshift-service-ca.crt\""
Jul 09 14:29:47.361403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.361383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-e75e34\"/\"default-dockercfg-qkb2f\""
Jul 09 14:29:47.370843 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.370821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:29:47.388243 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.388366 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388254    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.388366 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xdtm\" (UniqueName: \"kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.388366 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.388366 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.388523 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.388378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489412 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xdtm\" (UniqueName: \"kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489501    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489641 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489913 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489965 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.489965 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.489945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.492033 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.492011    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.492307 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.492287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.498910 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.498883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xdtm\" (UniqueName: \"kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm\") pod \"test-model-e75e34-kserve-d69f96c5f-wrxrj\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") " pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.667090 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.666985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:47.811263 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.811231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:29:47.813941 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:29:47.813913    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod331a3efd_6c50_49b1_b2e6_889cda4a1de4.slice/crio-a0de389577d968da5094a798c71fbcee8af3d1f173617a4abe42c159fd40a22e WatchSource:0}: Error finding container a0de389577d968da5094a798c71fbcee8af3d1f173617a4abe42c159fd40a22e: Status 404 returned error can't find the container with id a0de389577d968da5094a798c71fbcee8af3d1f173617a4abe42c159fd40a22e
Jul 09 14:29:47.949228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.949146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" event={"ID":"331a3efd-6c50-49b1-b2e6-889cda4a1de4","Type":"ContainerStarted","Data":"4fdc2347d74f02802f823226a56585f6b56c61047bae98efa6866a9ddb332e7d"}
Jul 09 14:29:47.949228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.949187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" event={"ID":"331a3efd-6c50-49b1-b2e6-889cda4a1de4","Type":"ContainerStarted","Data":"a0de389577d968da5094a798c71fbcee8af3d1f173617a4abe42c159fd40a22e"}
Jul 09 14:29:47.969380 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:47.969321    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" podStartSLOduration=0.969303669 podStartE2EDuration="969.303669ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:29:47.968095148 +0000 UTC m=+1893.509361930" watchObservedRunningTime="2026-07-09 14:29:47.969303669 +0000 UTC m=+1893.510570460"
Jul 09 14:29:57.668129 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:57.668086    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:57.668558 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:57.668147    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:57.680957 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:57.680929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:29:58.011609 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:29:58.011505    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:30:12.949953 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:12.949915    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:30:13.810430 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:13.810384    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:30:13.810769 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:13.810720    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" podUID="85c65c26-fe14-4857-966a-05867187e047" containerName="maas-api" containerID="cri-o://40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a" gracePeriod=30
Jul 09 14:30:14.270672 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.270638    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:30:14.351780 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.351684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls\") pod \"85c65c26-fe14-4857-966a-05867187e047\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") "
Jul 09 14:30:14.351932 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.351798    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbgtq\" (UniqueName: \"kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq\") pod \"85c65c26-fe14-4857-966a-05867187e047\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") "
Jul 09 14:30:14.351932 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.351844    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca\") pod \"85c65c26-fe14-4857-966a-05867187e047\" (UID: \"85c65c26-fe14-4857-966a-05867187e047\") "
Jul 09 14:30:14.352394 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.352359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "85c65c26-fe14-4857-966a-05867187e047" (UID: "85c65c26-fe14-4857-966a-05867187e047"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:30:14.354115 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.354080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "85c65c26-fe14-4857-966a-05867187e047" (UID: "85c65c26-fe14-4857-966a-05867187e047"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:30:14.354221 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.354169    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq" (OuterVolumeSpecName: "kube-api-access-nbgtq") pod "85c65c26-fe14-4857-966a-05867187e047" (UID: "85c65c26-fe14-4857-966a-05867187e047"). InnerVolumeSpecName "kube-api-access-nbgtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:30:14.453409 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.453372    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbgtq\" (UniqueName: \"kubernetes.io/projected/85c65c26-fe14-4857-966a-05867187e047-kube-api-access-nbgtq\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:14.453409 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.453402    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85c65c26-fe14-4857-966a-05867187e047-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:14.453409 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:14.453412    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85c65c26-fe14-4857-966a-05867187e047-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:15.084127 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.084096    2580 generic.go:356] "Generic (PLEG): container finished" podID="85c65c26-fe14-4857-966a-05867187e047" containerID="40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a" exitCode=0
Jul 09 14:30:15.084320 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.084149    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"
Jul 09 14:30:15.084320 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.084181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" event={"ID":"85c65c26-fe14-4857-966a-05867187e047","Type":"ContainerDied","Data":"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"}
Jul 09 14:30:15.084320 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.084249    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp" event={"ID":"85c65c26-fe14-4857-966a-05867187e047","Type":"ContainerDied","Data":"f619ada2b48677a065480426bc3b962bf8c1863871e022216ab81d88d6686542"}
Jul 09 14:30:15.084320 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.084280    2580 scope.go:117] "RemoveContainer" containerID="40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"
Jul 09 14:30:15.093919 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.093892    2580 scope.go:117] "RemoveContainer" containerID="40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"
Jul 09 14:30:15.094248 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:30:15.094230    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a\": container with ID starting with 40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a not found: ID does not exist" containerID="40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"
Jul 09 14:30:15.094294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.094258    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a"} err="failed to get container status \"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a\": rpc error: code = NotFound desc = could not find container \"40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a\": container with ID starting with 40cf54bb3b805217ea8e9dd857ada4eeb6fc14e7cf9cf6d30552016e28b6b33a not found: ID does not exist"
Jul 09 14:30:15.101964 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.101927    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:30:15.108755 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:15.108729    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a8a58b-59c67665f8-qcgxp"]
Jul 09 14:30:17.061088 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:17.061052    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="85c65c26-fe14-4857-966a-05867187e047" path="/var/lib/kubelet/pods/85c65c26-fe14-4857-966a-05867187e047/volumes"
Jul 09 14:30:27.972169 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:27.972126    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerName="main" containerID="cri-o://4fdc2347d74f02802f823226a56585f6b56c61047bae98efa6866a9ddb332e7d" gracePeriod=45
Jul 09 14:30:28.000959 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.000922    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.47:8000/ready\": dial tcp 10.132.0.47:8000: connect: connection refused"
Jul 09 14:30:28.150087 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.150056    2580 generic.go:356] "Generic (PLEG): container finished" podID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerID="4fdc2347d74f02802f823226a56585f6b56c61047bae98efa6866a9ddb332e7d" exitCode=0
Jul 09 14:30:28.150256 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.150126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" event={"ID":"331a3efd-6c50-49b1-b2e6-889cda4a1de4","Type":"ContainerDied","Data":"4fdc2347d74f02802f823226a56585f6b56c61047bae98efa6866a9ddb332e7d"}
Jul 09 14:30:28.221165 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.221141    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:30:28.274148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274073    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274119    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274147    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274180    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274228    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xdtm\" (UniqueName: \"kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274367 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274252    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home\") pod \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\" (UID: \"331a3efd-6c50-49b1-b2e6-889cda4a1de4\") "
Jul 09 14:30:28.274626 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache" (OuterVolumeSpecName: "model-cache") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:30:28.274626 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274400    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:30:28.274728 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274613    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home" (OuterVolumeSpecName: "home") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:30:28.274728 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274668    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-model-cache\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:28.274728 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.274686    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tmp-dir\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:28.276554 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.276530    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm" (OuterVolumeSpecName: "dshm") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:30:28.276554 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.276536    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm" (OuterVolumeSpecName: "kube-api-access-8xdtm") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "kube-api-access-8xdtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:30:28.276716 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.276636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "331a3efd-6c50-49b1-b2e6-889cda4a1de4" (UID: "331a3efd-6c50-49b1-b2e6-889cda4a1de4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:30:28.375140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.375090    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-dshm\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:28.375140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.375132    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/331a3efd-6c50-49b1-b2e6-889cda4a1de4-tls-certs\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:28.375140 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.375147    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xdtm\" (UniqueName: \"kubernetes.io/projected/331a3efd-6c50-49b1-b2e6-889cda4a1de4-kube-api-access-8xdtm\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:28.375400 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:28.375158    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/331a3efd-6c50-49b1-b2e6-889cda4a1de4-home\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:29.155908 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:29.155881    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"
Jul 09 14:30:29.156355 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:29.155920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj" event={"ID":"331a3efd-6c50-49b1-b2e6-889cda4a1de4","Type":"ContainerDied","Data":"a0de389577d968da5094a798c71fbcee8af3d1f173617a4abe42c159fd40a22e"}
Jul 09 14:30:29.156355 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:29.155970    2580 scope.go:117] "RemoveContainer" containerID="4fdc2347d74f02802f823226a56585f6b56c61047bae98efa6866a9ddb332e7d"
Jul 09 14:30:29.173531 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:29.173504    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:30:29.177809 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:29.177781    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-e75e34/test-model-e75e34-kserve-d69f96c5f-wrxrj"]
Jul 09 14:30:31.060591 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:31.060526    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" path="/var/lib/kubelet/pods/331a3efd-6c50-49b1-b2e6-889cda4a1de4/volumes"
Jul 09 14:30:57.034327 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.034287    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:30:57.034971 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.034750    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" podUID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" containerName="maas-api" containerID="cri-o://76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15" gracePeriod=30
Jul 09 14:30:57.286790 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.286726    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:30:57.290448 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.290415    2580 generic.go:356] "Generic (PLEG): container finished" podID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" containerID="76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15" exitCode=0
Jul 09 14:30:57.290599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.290458    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"
Jul 09 14:30:57.290599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.290492    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" event={"ID":"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8","Type":"ContainerDied","Data":"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"}
Jul 09 14:30:57.290599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.290582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q" event={"ID":"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8","Type":"ContainerDied","Data":"4a55252f1f4f09b1d2bc440a47b14706780d959747b8cf24bdc532c4ecda41ab"}
Jul 09 14:30:57.290599 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.290595    2580 scope.go:117] "RemoveContainer" containerID="76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"
Jul 09 14:30:57.298802 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.298782    2580 scope.go:117] "RemoveContainer" containerID="76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"
Jul 09 14:30:57.299044 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:30:57.299025    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15\": container with ID starting with 76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15 not found: ID does not exist" containerID="76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"
Jul 09 14:30:57.299096 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.299052    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15"} err="failed to get container status \"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15\": rpc error: code = NotFound desc = could not find container \"76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15\": container with ID starting with 76847baef49843e5432e7629a9367cd5cec40fe52ead757b7b0985aabab93e15 not found: ID does not exist"
Jul 09 14:30:57.447797 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.447759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") pod \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") "
Jul 09 14:30:57.448010 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.447845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca\") pod \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") "
Jul 09 14:30:57.448010 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.447898    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6qfc\" (UniqueName: \"kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc\") pod \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\" (UID: \"4ef1f501-a9f3-4cc3-8910-d1cb763ffed8\") "
Jul 09 14:30:57.448253 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.448226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" (UID: "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:30:57.450200 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.450174    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" (UID: "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:30:57.450329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.450232    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc" (OuterVolumeSpecName: "kube-api-access-k6qfc") pod "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" (UID: "4ef1f501-a9f3-4cc3-8910-d1cb763ffed8"). InnerVolumeSpecName "kube-api-access-k6qfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:30:57.548956 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.548869    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:57.548956 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.548897    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6qfc\" (UniqueName: \"kubernetes.io/projected/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-kube-api-access-k6qfc\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:57.548956 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.548906    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:30:57.611893 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.611868    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:30:57.616233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:57.616210    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-e75e34-5764f9bd47-4857q"]
Jul 09 14:30:59.060878 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:30:59.060842    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" path="/var/lib/kubelet/pods/4ef1f501-a9f3-4cc3-8910-d1cb763ffed8/volumes"
Jul 09 14:31:07.324471 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324354    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324735    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerName="main"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerName="main"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324770    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324775    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324784    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="85c65c26-fe14-4857-966a-05867187e047" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324790    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="85c65c26-fe14-4857-966a-05867187e047" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324849    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="85c65c26-fe14-4857-966a-05867187e047" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324856    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ef1f501-a9f3-4cc3-8910-d1cb763ffed8" containerName="maas-api"
Jul 09 14:31:07.324962 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.324862    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="331a3efd-6c50-49b1-b2e6-889cda4a1de4" containerName="main"
Jul 09 14:31:07.329191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.329162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.332247 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.332227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-eeeac1-serving-cert\""
Jul 09 14:31:07.348381 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.348344    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:07.441474 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.441426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.441741 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.441487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.441741 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.441529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzrw4\" (UniqueName: \"kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.542193 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.542146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.542193 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.542201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.542482 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.542223    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzrw4\" (UniqueName: \"kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.542482 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:07.542323    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-serving-cert: secret "maas-api-e2e-api-a-eeeac1-serving-cert" not found
Jul 09 14:31:07.542482 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:07.542415    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls podName:e467f3a9-4d78-4a00-9a5c-9a8b81380d43 nodeName:}" failed. No retries permitted until 2026-07-09 14:31:08.042393865 +0000 UTC m=+1973.583660627 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls") pod "maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" (UID: "e467f3a9-4d78-4a00-9a5c-9a8b81380d43") : secret "maas-api-e2e-api-a-eeeac1-serving-cert" not found
Jul 09 14:31:07.542854 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.542832    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:07.556240 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:07.556214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzrw4\" (UniqueName: \"kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:08.048400 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:08.048362    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:08.051184 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:08.051149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") pod \"maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:08.248612 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:08.248547    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:08.410245 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:08.410213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:08.411021 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:31:08.410987    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode467f3a9_4d78_4a00_9a5c_9a8b81380d43.slice/crio-65db8d04e3eb75316004cd9efd7e127057f56d0b0b6334e27c94f143f6dcc4f0 WatchSource:0}: Error finding container 65db8d04e3eb75316004cd9efd7e127057f56d0b0b6334e27c94f143f6dcc4f0: Status 404 returned error can't find the container with id 65db8d04e3eb75316004cd9efd7e127057f56d0b0b6334e27c94f143f6dcc4f0
Jul 09 14:31:09.361536 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:09.361497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" event={"ID":"e467f3a9-4d78-4a00-9a5c-9a8b81380d43","Type":"ContainerStarted","Data":"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"}
Jul 09 14:31:09.361760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:09.361556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" event={"ID":"e467f3a9-4d78-4a00-9a5c-9a8b81380d43","Type":"ContainerStarted","Data":"65db8d04e3eb75316004cd9efd7e127057f56d0b0b6334e27c94f143f6dcc4f0"}
Jul 09 14:31:09.361760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:09.361599    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:09.383761 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:09.383701    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" podStartSLOduration=1.940003396 podStartE2EDuration="2.383680224s" totalImagesPullingTime="443.676828ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:09.381623082 +0000 UTC m=+1974.922889892" watchObservedRunningTime="2026-07-09 14:31:09.383680224 +0000 UTC m=+1974.924947007"
Jul 09 14:31:15.373448 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.373419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:15.484024 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.483925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:15.489334 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.489303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.493108 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.493084    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-c02955-serving-cert\""
Jul 09 14:31:15.507660 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.507629    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:15.508101 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.508064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.508365 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.508330    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7kbw\" (UniqueName: \"kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.508496 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.508442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.609614 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.609552    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.609791 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.609671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.609791 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:15.609710    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-serving-cert: secret "maas-api-e2e-api-b-c02955-serving-cert" not found
Jul 09 14:31:15.609791 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.609729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7kbw\" (UniqueName: \"kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.609791 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:15.609786    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls podName:11d5d2b5-363e-4724-b38b-c7ce0179e7d1 nodeName:}" failed. No retries permitted until 2026-07-09 14:31:16.109762377 +0000 UTC m=+1981.651029155 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls") pod "maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" (UID: "11d5d2b5-363e-4724-b38b-c7ce0179e7d1") : secret "maas-api-e2e-api-b-c02955-serving-cert" not found
Jul 09 14:31:15.610383 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.610363    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:15.620892 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:15.620852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7kbw\" (UniqueName: \"kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:16.114767 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:16.114722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:16.117593 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:16.117536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") pod \"maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:16.400158 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:16.400055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:16.554811 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:16.554782    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:16.557439 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:31:16.557410    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d5d2b5_363e_4724_b38b_c7ce0179e7d1.slice/crio-db3feb5ecdb397e7350055c082bc564b0c774d716fad7bbad0f81c343b17cae5 WatchSource:0}: Error finding container db3feb5ecdb397e7350055c082bc564b0c774d716fad7bbad0f81c343b17cae5: Status 404 returned error can't find the container with id db3feb5ecdb397e7350055c082bc564b0c774d716fad7bbad0f81c343b17cae5
Jul 09 14:31:17.412255 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:17.412214    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" event={"ID":"11d5d2b5-363e-4724-b38b-c7ce0179e7d1","Type":"ContainerStarted","Data":"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"}
Jul 09 14:31:17.412684 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:17.412298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" event={"ID":"11d5d2b5-363e-4724-b38b-c7ce0179e7d1","Type":"ContainerStarted","Data":"db3feb5ecdb397e7350055c082bc564b0c774d716fad7bbad0f81c343b17cae5"}
Jul 09 14:31:17.412684 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:17.412352    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:17.433156 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:17.433102    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" podStartSLOduration=2.046604218 podStartE2EDuration="2.433082879s" totalImagesPullingTime="386.478661ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:17.43188194 +0000 UTC m=+1982.973148751" watchObservedRunningTime="2026-07-09 14:31:17.433082879 +0000 UTC m=+1982.974349662"
Jul 09 14:31:23.423273 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:23.423245    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:28.840370 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:28.840332    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:28.841051 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:28.840560    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" podUID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" containerName="maas-api" containerID="cri-o://970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538" gracePeriod=30
Jul 09 14:31:29.102396 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.102325    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:29.237171 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.237134    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca\") pod \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") "
Jul 09 14:31:29.237171 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.237186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") pod \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") "
Jul 09 14:31:29.237451 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.237268    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zzrw4\" (UniqueName: \"kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4\") pod \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\" (UID: \"e467f3a9-4d78-4a00-9a5c-9a8b81380d43\") "
Jul 09 14:31:29.237666 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.237632    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e467f3a9-4d78-4a00-9a5c-9a8b81380d43" (UID: "e467f3a9-4d78-4a00-9a5c-9a8b81380d43"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:31:29.239613 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.239561    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e467f3a9-4d78-4a00-9a5c-9a8b81380d43" (UID: "e467f3a9-4d78-4a00-9a5c-9a8b81380d43"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:31:29.239711 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.239632    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4" (OuterVolumeSpecName: "kube-api-access-zzrw4") pod "e467f3a9-4d78-4a00-9a5c-9a8b81380d43" (UID: "e467f3a9-4d78-4a00-9a5c-9a8b81380d43"). InnerVolumeSpecName "kube-api-access-zzrw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:31:29.338906 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.338857    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:29.338906 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.338889    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:29.338906 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.338900    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zzrw4\" (UniqueName: \"kubernetes.io/projected/e467f3a9-4d78-4a00-9a5c-9a8b81380d43-kube-api-access-zzrw4\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:29.468857 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.468748    2580 generic.go:356] "Generic (PLEG): container finished" podID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" containerID="970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538" exitCode=0
Jul 09 14:31:29.468857 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.468796    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"
Jul 09 14:31:29.468857 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.468834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" event={"ID":"e467f3a9-4d78-4a00-9a5c-9a8b81380d43","Type":"ContainerDied","Data":"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"}
Jul 09 14:31:29.469167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.468900    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js" event={"ID":"e467f3a9-4d78-4a00-9a5c-9a8b81380d43","Type":"ContainerDied","Data":"65db8d04e3eb75316004cd9efd7e127057f56d0b0b6334e27c94f143f6dcc4f0"}
Jul 09 14:31:29.469167 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.468934    2580 scope.go:117] "RemoveContainer" containerID="970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"
Jul 09 14:31:29.478236 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.478219    2580 scope.go:117] "RemoveContainer" containerID="970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"
Jul 09 14:31:29.478558 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:29.478537    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538\": container with ID starting with 970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538 not found: ID does not exist" containerID="970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"
Jul 09 14:31:29.478636 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.478599    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538"} err="failed to get container status \"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538\": rpc error: code = NotFound desc = could not find container \"970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538\": container with ID starting with 970916a5e54fa9f3e0e9628eea93ada7ec2e577c5390d505ce5697aa5d05f538 not found: ID does not exist"
Jul 09 14:31:29.491879 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.491844    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:29.498817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:29.498788    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-eeeac1-6dc8dd585d-267js"]
Jul 09 14:31:31.060584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:31.060538    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" path="/var/lib/kubelet/pods/e467f3a9-4d78-4a00-9a5c-9a8b81380d43/volumes"
Jul 09 14:31:35.901705 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:35.901672    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:35.902160 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:35.901871    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" podUID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" containerName="maas-api" containerID="cri-o://371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329" gracePeriod=30
Jul 09 14:31:36.159297 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.159221    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:36.307417 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.307365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") pod \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") "
Jul 09 14:31:36.307633 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.307487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca\") pod \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") "
Jul 09 14:31:36.307633 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.307542    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7kbw\" (UniqueName: \"kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw\") pod \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\" (UID: \"11d5d2b5-363e-4724-b38b-c7ce0179e7d1\") "
Jul 09 14:31:36.307905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.307878    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11d5d2b5-363e-4724-b38b-c7ce0179e7d1" (UID: "11d5d2b5-363e-4724-b38b-c7ce0179e7d1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:31:36.309832 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.309802    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw" (OuterVolumeSpecName: "kube-api-access-c7kbw") pod "11d5d2b5-363e-4724-b38b-c7ce0179e7d1" (UID: "11d5d2b5-363e-4724-b38b-c7ce0179e7d1"). InnerVolumeSpecName "kube-api-access-c7kbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:31:36.309957 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.309838    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11d5d2b5-363e-4724-b38b-c7ce0179e7d1" (UID: "11d5d2b5-363e-4724-b38b-c7ce0179e7d1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:31:36.408294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.408251    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:36.408294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.408285    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:36.408294 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.408301    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7kbw\" (UniqueName: \"kubernetes.io/projected/11d5d2b5-363e-4724-b38b-c7ce0179e7d1-kube-api-access-c7kbw\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:31:36.503696 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.503605    2580 generic.go:356] "Generic (PLEG): container finished" podID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" containerID="371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329" exitCode=0
Jul 09 14:31:36.503846 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.503716    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"
Jul 09 14:31:36.503846 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.503734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" event={"ID":"11d5d2b5-363e-4724-b38b-c7ce0179e7d1","Type":"ContainerDied","Data":"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"}
Jul 09 14:31:36.503846 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.503812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9" event={"ID":"11d5d2b5-363e-4724-b38b-c7ce0179e7d1","Type":"ContainerDied","Data":"db3feb5ecdb397e7350055c082bc564b0c774d716fad7bbad0f81c343b17cae5"}
Jul 09 14:31:36.503846 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.503834    2580 scope.go:117] "RemoveContainer" containerID="371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"
Jul 09 14:31:36.513070 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.513050    2580 scope.go:117] "RemoveContainer" containerID="371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"
Jul 09 14:31:36.513334 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:36.513311    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329\": container with ID starting with 371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329 not found: ID does not exist" containerID="371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"
Jul 09 14:31:36.513396 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.513341    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329"} err="failed to get container status \"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329\": rpc error: code = NotFound desc = could not find container \"371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329\": container with ID starting with 371fa90c0f6122ae2bab8cc70cb78fb9187e18f55f32a5882162b6590199a329 not found: ID does not exist"
Jul 09 14:31:36.528177 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.528144    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:36.530459 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:36.530431    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-c02955-5cc8bfd4c9-rwcp9"]
Jul 09 14:31:37.060186 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:37.060153    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" path="/var/lib/kubelet/pods/11d5d2b5-363e-4724-b38b-c7ce0179e7d1/volumes"
Jul 09 14:31:46.206946 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.206914    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:31:46.207329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207276    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" containerName="maas-api"
Jul 09 14:31:46.207329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207287    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" containerName="maas-api"
Jul 09 14:31:46.207329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207300    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" containerName="maas-api"
Jul 09 14:31:46.207329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207306    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" containerName="maas-api"
Jul 09 14:31:46.207462 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207356    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e467f3a9-4d78-4a00-9a5c-9a8b81380d43" containerName="maas-api"
Jul 09 14:31:46.207462 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.207365    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d5d2b5-363e-4724-b38b-c7ce0179e7d1" containerName="maas-api"
Jul 09 14:31:46.211644 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.211622    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.216584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.216540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-5a8685-serving-cert\""
Jul 09 14:31:46.230331 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.230301    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:31:46.300517 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.300474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.300741 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.300542    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.300741 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.300647    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnlcx\" (UniqueName: \"kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.401549 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.401504    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.401782 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.401674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.401782 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.401737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnlcx\" (UniqueName: \"kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.401883 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:46.401826    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-serving-cert: secret "maas-api-e2e-shared-a-5a8685-serving-cert" not found
Jul 09 14:31:46.401935 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:31:46.401921    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls podName:b333976f-1506-4f6b-a39b-77e24d73e1e5 nodeName:}" failed. No retries permitted until 2026-07-09 14:31:46.90189892 +0000 UTC m=+2012.443165680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls") pod "maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" (UID: "b333976f-1506-4f6b-a39b-77e24d73e1e5") : secret "maas-api-e2e-shared-a-5a8685-serving-cert" not found
Jul 09 14:31:46.402175 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.402158    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.414530 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.414487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnlcx\" (UniqueName: \"kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.906668 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.906628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:46.909261 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:46.909230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") pod \"maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:47.123344 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:47.123302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:47.280682 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:47.280652    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:31:47.282738 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:31:47.282710    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb333976f_1506_4f6b_a39b_77e24d73e1e5.slice/crio-6c214969bb15ffe6eeea0bc3f7ec62ea68c40055e4e8f7bf7c3d6c253f90d9b9 WatchSource:0}: Error finding container 6c214969bb15ffe6eeea0bc3f7ec62ea68c40055e4e8f7bf7c3d6c253f90d9b9: Status 404 returned error can't find the container with id 6c214969bb15ffe6eeea0bc3f7ec62ea68c40055e4e8f7bf7c3d6c253f90d9b9
Jul 09 14:31:47.556348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:47.556303    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" event={"ID":"b333976f-1506-4f6b-a39b-77e24d73e1e5","Type":"ContainerStarted","Data":"6c214969bb15ffe6eeea0bc3f7ec62ea68c40055e4e8f7bf7c3d6c253f90d9b9"}
Jul 09 14:31:48.562122 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:48.562085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" event={"ID":"b333976f-1506-4f6b-a39b-77e24d73e1e5","Type":"ContainerStarted","Data":"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"}
Jul 09 14:31:48.562588 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:48.562225    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:48.590944 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:48.590889    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" podStartSLOduration=2.05368038 podStartE2EDuration="2.590871143s" totalImagesPullingTime="537.190763ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:48.589076972 +0000 UTC m=+2014.130343756" watchObservedRunningTime="2026-07-09 14:31:48.590871143 +0000 UTC m=+2014.132137925"
Jul 09 14:31:53.517504 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.517460    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:31:53.521655 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.521625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.524751 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.524716    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-a4b0c9-serving-cert\""
Jul 09 14:31:53.535519 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.535179    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:31:53.566833 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.566795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.566988 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.566861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.566988 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.566924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrshj\" (UniqueName: \"kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.667525 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.667482    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrshj\" (UniqueName: \"kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.667771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.667608    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.667771 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.667683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.668329 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.668308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.670357 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.670340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.677085 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.677059    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrshj\" (UniqueName: \"kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj\") pod \"maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:53.840745 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:53.840692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:54.002322 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.002295    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:31:54.006077 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:31:54.006007    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bef7f3f_d87e_42d0_9eef_dafeaf407f5c.slice/crio-40dac22fd5235b165c1b743128b34aa29c36aae73a39bd1929d312b23d7877f8 WatchSource:0}: Error finding container 40dac22fd5235b165c1b743128b34aa29c36aae73a39bd1929d312b23d7877f8: Status 404 returned error can't find the container with id 40dac22fd5235b165c1b743128b34aa29c36aae73a39bd1929d312b23d7877f8
Jul 09 14:31:54.573456 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.573426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:31:54.594009 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.593961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" event={"ID":"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c","Type":"ContainerStarted","Data":"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"}
Jul 09 14:31:54.594192 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.594032    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" event={"ID":"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c","Type":"ContainerStarted","Data":"40dac22fd5235b165c1b743128b34aa29c36aae73a39bd1929d312b23d7877f8"}
Jul 09 14:31:54.594192 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.594138    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:31:54.618862 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:31:54.618705    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" podStartSLOduration=1.223341583 podStartE2EDuration="1.618680851s" totalImagesPullingTime="395.339268ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:31:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:31:54.618297635 +0000 UTC m=+2020.159564435" watchObservedRunningTime="2026-07-09 14:31:54.618680851 +0000 UTC m=+2020.159947636"
Jul 09 14:32:00.604817 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:00.604787    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:32:32.443149 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.443107    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:32:32.446830 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.446804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.453117 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.453080    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a4b0c9\"/\"auth-test-model-a4b0c9-kserve-self-signed-certs\""
Jul 09 14:32:32.453708 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.453684    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a4b0c9\"/\"default-dockercfg-cg567\""
Jul 09 14:32:32.453876 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.453833    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a4b0c9\"/\"openshift-service-ca.crt\""
Jul 09 14:32:32.453995 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.453892    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a4b0c9\"/\"kube-root-ca.crt\""
Jul 09 14:32:32.458464 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.458437    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:32:32.627205 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.627418 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.627418 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.627418 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627333    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brssz\" (UniqueName: \"kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.627418 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.627614 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.627451    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728145 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brssz\" (UniqueName: \"kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728180    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728498 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728691 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728743 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728706    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.728782 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.728754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.730728 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.730703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.730918 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.730898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.738083 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.738053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brssz\" (UniqueName: \"kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz\") pod \"auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") " pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.765809 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.765773    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:32.909651 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:32.909620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:32:32.912540 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:32:32.912513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd0b30ebf_8163_4e5f_b3b0_cdf2ad0e2f34.slice/crio-ccd12f5cf13a5bcb83e49726c006bc72d2d3f25cea60d26afd334e02926bc161 WatchSource:0}: Error finding container ccd12f5cf13a5bcb83e49726c006bc72d2d3f25cea60d26afd334e02926bc161: Status 404 returned error can't find the container with id ccd12f5cf13a5bcb83e49726c006bc72d2d3f25cea60d26afd334e02926bc161
Jul 09 14:32:33.791090 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:33.791052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" event={"ID":"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34","Type":"ContainerStarted","Data":"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"}
Jul 09 14:32:33.791489 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:33.791102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" event={"ID":"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34","Type":"ContainerStarted","Data":"ccd12f5cf13a5bcb83e49726c006bc72d2d3f25cea60d26afd334e02926bc161"}
Jul 09 14:32:33.808949 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:33.808896    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" podStartSLOduration=1.808878666 podStartE2EDuration="1.808878666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:32:33.80820879 +0000 UTC m=+2059.349475618" watchObservedRunningTime="2026-07-09 14:32:33.808878666 +0000 UTC m=+2059.350145449"
Jul 09 14:32:42.765992 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:42.765886    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:42.765992 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:42.765964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:42.778639 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:42.778608    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:32:42.850712 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:32:42.850685    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:33:15.075255 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:33:15.075226    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:33:15.080841 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:33:15.080814    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:33:15.082152 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:33:15.082126    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:33:15.087406 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:33:15.087386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:34:33.326413 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:33.326326    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:34:48.347638 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.347559    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" podUID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" containerName="main" containerID="cri-o://d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f" gracePeriod=45
Jul 09 14:34:48.614649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.614617    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.661926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.662000    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-brssz\" (UniqueName: \"kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.662029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.662055    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.662095    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.662145    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm\") pod \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\" (UID: \"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34\") "
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.665837    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home" (OuterVolumeSpecName: "home") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.666012    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache" (OuterVolumeSpecName: "model-cache") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:34:48.666453 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.666156    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:34:48.672751 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.672692    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm" (OuterVolumeSpecName: "dshm") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:34:48.674809 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.674731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz" (OuterVolumeSpecName: "kube-api-access-brssz") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "kube-api-access-brssz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:34:48.693596 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.687711    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" (UID: "d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:34:48.763402 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763363    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-model-cache\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:48.763402 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763396    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-dshm\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:48.763402 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763405    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tmp-dir\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:48.763667 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763415    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-brssz\" (UniqueName: \"kubernetes.io/projected/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-kube-api-access-brssz\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:48.763667 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763424    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-tls-certs\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:48.763667 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:48.763432    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34-home\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:34:49.458916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.458810    2580 generic.go:356] "Generic (PLEG): container finished" podID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" containerID="d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f" exitCode=0
Jul 09 14:34:49.458916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.458850    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"
Jul 09 14:34:49.458916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.458855    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" event={"ID":"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34","Type":"ContainerDied","Data":"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"}
Jul 09 14:34:49.458916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.458896    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42" event={"ID":"d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34","Type":"ContainerDied","Data":"ccd12f5cf13a5bcb83e49726c006bc72d2d3f25cea60d26afd334e02926bc161"}
Jul 09 14:34:49.458916 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.458919    2580 scope.go:117] "RemoveContainer" containerID="d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"
Jul 09 14:34:49.468040 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.468019    2580 scope.go:117] "RemoveContainer" containerID="d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"
Jul 09 14:34:49.468347 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:34:49.468326    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f\": container with ID starting with d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f not found: ID does not exist" containerID="d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"
Jul 09 14:34:49.468424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.468355    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f"} err="failed to get container status \"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f\": rpc error: code = NotFound desc = could not find container \"d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f\": container with ID starting with d847a53cf866317fdf764777a85da45c39c4a959f7f1b7ac06c93228d6ca7a5f not found: ID does not exist"
Jul 09 14:34:49.476205 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.476153    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:34:49.478422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:49.478400    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a4b0c9/auth-test-model-a4b0c9-kserve-6885dbbc9f-nkm42"]
Jul 09 14:34:51.060927 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:34:51.060885    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" path="/var/lib/kubelet/pods/d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34/volumes"
Jul 09 14:35:55.635100 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.635064    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:35:55.635649 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.635633    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" containerName="main"
Jul 09 14:35:55.635701 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.635652    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" containerName="main"
Jul 09 14:35:55.635777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.635766    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d0b30ebf-8163-4e5f-b3b0-cdf2ad0e2f34" containerName="main"
Jul 09 14:35:55.639251 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.639231    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.642085 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.642050    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5a8685\"/\"openshift-service-ca.crt\""
Jul 09 14:35:55.643004 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.642981    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5a8685\"/\"kube-root-ca.crt\""
Jul 09 14:35:55.643141 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.642995    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5a8685\"/\"rate-test-model-5a8685-kserve-self-signed-certs\""
Jul 09 14:35:55.643141 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.643079    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5a8685\"/\"default-dockercfg-vj4tv\""
Jul 09 14:35:55.648644 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.648620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:35:55.682207 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k88dk\" (UniqueName: \"kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.682207 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682212    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.682437 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.682437 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.682437 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682337    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.682437 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.682363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k88dk\" (UniqueName: \"kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783652 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783707 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.783707 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.783687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.785708 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.785683    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.785978 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.785963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.791621 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.791596    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k88dk\" (UniqueName: \"kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk\") pod \"rate-test-model-5a8685-kserve-95b84fd5f-wbnkv\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") " pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:55.952105 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:55.952008    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:35:56.094233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:56.094205    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:35:56.097116 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:35:56.097087    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09c62251_ef9a_447d_a050_0d1881f2b90c.slice/crio-8724e067d4dcd5bd25c8a25bab093009b01f7007d46f8d5f8dfebbbd1b92df38 WatchSource:0}: Error finding container 8724e067d4dcd5bd25c8a25bab093009b01f7007d46f8d5f8dfebbbd1b92df38: Status 404 returned error can't find the container with id 8724e067d4dcd5bd25c8a25bab093009b01f7007d46f8d5f8dfebbbd1b92df38
Jul 09 14:35:56.788449 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:56.788412    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" event={"ID":"09c62251-ef9a-447d-a050-0d1881f2b90c","Type":"ContainerStarted","Data":"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"}
Jul 09 14:35:56.788957 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:56.788458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" event={"ID":"09c62251-ef9a-447d-a050-0d1881f2b90c","Type":"ContainerStarted","Data":"8724e067d4dcd5bd25c8a25bab093009b01f7007d46f8d5f8dfebbbd1b92df38"}
Jul 09 14:35:56.807669 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:35:56.807619    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" podStartSLOduration=1.807602686 podStartE2EDuration="1.807602686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 14:35:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:35:56.805612971 +0000 UTC m=+2262.346879767" watchObservedRunningTime="2026-07-09 14:35:56.807602686 +0000 UTC m=+2262.348869470"
Jul 09 14:36:05.952979 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:36:05.952929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:36:05.953411 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:36:05.953009    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:36:05.966072 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:36:05.966035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:36:06.850055 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:36:06.850021    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:37:21.763106 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:21.763023    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:37:23.644433 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.644386    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7315-140081359833904"]
Jul 09 14:37:23.647911 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.647887    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:23.650740 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.650712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-4lxwr\""
Jul 09 14:37:23.653670 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.653644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359833904"]
Jul 09 14:37:23.716398 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.716352    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9b6j\" (UniqueName: \"kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j\") pod \"test-curl-7315-140081359833904\" (UID: \"3dea1f37-294d-4289-a1c9-447454c67e87\") " pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:23.817225 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.817175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9b6j\" (UniqueName: \"kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j\") pod \"test-curl-7315-140081359833904\" (UID: \"3dea1f37-294d-4289-a1c9-447454c67e87\") " pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:23.826383 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.826346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9b6j\" (UniqueName: \"kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j\") pod \"test-curl-7315-140081359833904\" (UID: \"3dea1f37-294d-4289-a1c9-447454c67e87\") " pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:23.958500 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:23.958397    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:24.103821 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:24.103786    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359833904"]
Jul 09 14:37:24.106549 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:37:24.106509    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3dea1f37_294d_4289_a1c9_447454c67e87.slice/crio-a5488b049d40d517485d5e2c02c116fb1bf40662b9cb3130877d205b18fb126e WatchSource:0}: Error finding container a5488b049d40d517485d5e2c02c116fb1bf40662b9cb3130877d205b18fb126e: Status 404 returned error can't find the container with id a5488b049d40d517485d5e2c02c116fb1bf40662b9cb3130877d205b18fb126e
Jul 09 14:37:24.107907 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:24.107888    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 14:37:24.203478 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:24.203436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359833904" event={"ID":"3dea1f37-294d-4289-a1c9-447454c67e87","Type":"ContainerStarted","Data":"a5488b049d40d517485d5e2c02c116fb1bf40662b9cb3130877d205b18fb126e"}
Jul 09 14:37:25.210085 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:25.210044    2580 generic.go:356] "Generic (PLEG): container finished" podID="3dea1f37-294d-4289-a1c9-447454c67e87" containerID="bddf9557b896c6545c8d8b11fe709eff406f23e37df699a56c3e2eb574c4d4d0" exitCode=0
Jul 09 14:37:25.210503 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:25.210085    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359833904" event={"ID":"3dea1f37-294d-4289-a1c9-447454c67e87","Type":"ContainerDied","Data":"bddf9557b896c6545c8d8b11fe709eff406f23e37df699a56c3e2eb574c4d4d0"}
Jul 09 14:37:26.355308 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.355283    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:26.389620 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.389583    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7315-140081359833904_3dea1f37-294d-4289-a1c9-447454c67e87/test-curl-7315-140081359833904/0.log"
Jul 09 14:37:26.413156 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.413110    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7315-140081359833904"]
Jul 09 14:37:26.415287 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.415259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7315-140081359833904"]
Jul 09 14:37:26.442878 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.442850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9b6j\" (UniqueName: \"kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j\") pod \"3dea1f37-294d-4289-a1c9-447454c67e87\" (UID: \"3dea1f37-294d-4289-a1c9-447454c67e87\") "
Jul 09 14:37:26.445212 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.445182    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j" (OuterVolumeSpecName: "kube-api-access-h9b6j") pod "3dea1f37-294d-4289-a1c9-447454c67e87" (UID: "3dea1f37-294d-4289-a1c9-447454c67e87"). InnerVolumeSpecName "kube-api-access-h9b6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:26.543960 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:26.543870    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9b6j\" (UniqueName: \"kubernetes.io/projected/3dea1f37-294d-4289-a1c9-447454c67e87-kube-api-access-h9b6j\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:27.060218 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:27.060185    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3dea1f37-294d-4289-a1c9-447454c67e87" path="/var/lib/kubelet/pods/3dea1f37-294d-4289-a1c9-447454c67e87/volumes"
Jul 09 14:37:27.220870 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:27.220841    2580 scope.go:117] "RemoveContainer" containerID="bddf9557b896c6545c8d8b11fe709eff406f23e37df699a56c3e2eb574c4d4d0"
Jul 09 14:37:27.220870 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:27.220856    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359833904"
Jul 09 14:37:32.505384 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.505345    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7315-140081359760096"]
Jul 09 14:37:32.505821 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.505749    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3dea1f37-294d-4289-a1c9-447454c67e87" containerName="test-curl-7315-140081359833904"
Jul 09 14:37:32.505821 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.505762    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3dea1f37-294d-4289-a1c9-447454c67e87" containerName="test-curl-7315-140081359833904"
Jul 09 14:37:32.505821 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.505818    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3dea1f37-294d-4289-a1c9-447454c67e87" containerName="test-curl-7315-140081359833904"
Jul 09 14:37:32.510435 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.510415    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:32.513185 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.513162    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-4lxwr\""
Jul 09 14:37:32.514533 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.514509    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760096"]
Jul 09 14:37:32.601211 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.601181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnzb5\" (UniqueName: \"kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5\") pod \"test-curl-7315-140081359760096\" (UID: \"099e61a3-6b8b-4792-b821-5017eb456a65\") " pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:32.702325 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.702290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnzb5\" (UniqueName: \"kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5\") pod \"test-curl-7315-140081359760096\" (UID: \"099e61a3-6b8b-4792-b821-5017eb456a65\") " pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:32.711022 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.710987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnzb5\" (UniqueName: \"kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5\") pod \"test-curl-7315-140081359760096\" (UID: \"099e61a3-6b8b-4792-b821-5017eb456a65\") " pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:32.821333 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.821301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:32.956177 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:32.956150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760096"]
Jul 09 14:37:32.958302 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:37:32.958275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod099e61a3_6b8b_4792_b821_5017eb456a65.slice/crio-7be0db62da7e33d807249298edeef9f217e0c7a47dd1d51f89321830715d316b WatchSource:0}: Error finding container 7be0db62da7e33d807249298edeef9f217e0c7a47dd1d51f89321830715d316b: Status 404 returned error can't find the container with id 7be0db62da7e33d807249298edeef9f217e0c7a47dd1d51f89321830715d316b
Jul 09 14:37:33.251153 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:33.251025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359760096" event={"ID":"099e61a3-6b8b-4792-b821-5017eb456a65","Type":"ContainerStarted","Data":"7be0db62da7e33d807249298edeef9f217e0c7a47dd1d51f89321830715d316b"}
Jul 09 14:37:34.256713 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:34.256676    2580 generic.go:356] "Generic (PLEG): container finished" podID="099e61a3-6b8b-4792-b821-5017eb456a65" containerID="c5330455e4b81e98ba18be311c5cdeeebb87da43bbc1b3ef324ea26a1093970c" exitCode=0
Jul 09 14:37:34.257150 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:34.256769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359760096" event={"ID":"099e61a3-6b8b-4792-b821-5017eb456a65","Type":"ContainerDied","Data":"c5330455e4b81e98ba18be311c5cdeeebb87da43bbc1b3ef324ea26a1093970c"}
Jul 09 14:37:35.407274 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.407250    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:35.428084 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.428054    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7315-140081359760096_099e61a3-6b8b-4792-b821-5017eb456a65/test-curl-7315-140081359760096/0.log"
Jul 09 14:37:35.451406 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.451367    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760096"]
Jul 09 14:37:35.455935 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.455901    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760096"]
Jul 09 14:37:35.528920 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.528811    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnzb5\" (UniqueName: \"kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5\") pod \"099e61a3-6b8b-4792-b821-5017eb456a65\" (UID: \"099e61a3-6b8b-4792-b821-5017eb456a65\") "
Jul 09 14:37:35.531045 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.531020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5" (OuterVolumeSpecName: "kube-api-access-xnzb5") pod "099e61a3-6b8b-4792-b821-5017eb456a65" (UID: "099e61a3-6b8b-4792-b821-5017eb456a65"). InnerVolumeSpecName "kube-api-access-xnzb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:35.581264 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.581230    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7315-140081359760816"]
Jul 09 14:37:35.581747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.581726    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" containerName="test-curl-7315-140081359760096"
Jul 09 14:37:35.581747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.581747    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" containerName="test-curl-7315-140081359760096"
Jul 09 14:37:35.581939 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.581867    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" containerName="test-curl-7315-140081359760096"
Jul 09 14:37:35.584943 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.584925    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:35.589915 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.589890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760816"]
Jul 09 14:37:35.629600 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.629520    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7n9j\" (UniqueName: \"kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j\") pod \"test-curl-7315-140081359760816\" (UID: \"194b4e14-3ebd-424e-bdd9-f6a50f133910\") " pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:35.629805 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.629742    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnzb5\" (UniqueName: \"kubernetes.io/projected/099e61a3-6b8b-4792-b821-5017eb456a65-kube-api-access-xnzb5\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:35.731207 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.731160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7n9j\" (UniqueName: \"kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j\") pod \"test-curl-7315-140081359760816\" (UID: \"194b4e14-3ebd-424e-bdd9-f6a50f133910\") " pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:35.740921 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.740890    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7n9j\" (UniqueName: \"kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j\") pod \"test-curl-7315-140081359760816\" (UID: \"194b4e14-3ebd-424e-bdd9-f6a50f133910\") " pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:35.896173 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:35.896130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:36.035300 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.035273    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760816"]
Jul 09 14:37:36.037523 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:37:36.037492    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod194b4e14_3ebd_424e_bdd9_f6a50f133910.slice/crio-4869de05383ceb113ca2e298d959aa3ad1a9ad46be520ce6c32579086632c24e WatchSource:0}: Error finding container 4869de05383ceb113ca2e298d959aa3ad1a9ad46be520ce6c32579086632c24e: Status 404 returned error can't find the container with id 4869de05383ceb113ca2e298d959aa3ad1a9ad46be520ce6c32579086632c24e
Jul 09 14:37:36.267763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.267661    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7be0db62da7e33d807249298edeef9f217e0c7a47dd1d51f89321830715d316b"
Jul 09 14:37:36.267763 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.267683    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:36.269032 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.268997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359760816" event={"ID":"194b4e14-3ebd-424e-bdd9-f6a50f133910","Type":"ContainerStarted","Data":"4869de05383ceb113ca2e298d959aa3ad1a9ad46be520ce6c32579086632c24e"}
Jul 09 14:37:36.278925 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:37:36.278890    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7315-140081359760096\" is forbidden: User \"system:node:ip-10-0-143-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-235.ec2.internal' and this object" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:36.784656 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.784612    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerName="main" containerID="cri-o://de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9" gracePeriod=45
Jul 09 14:37:36.839593 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:36.839537    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerName="main" probeResult="failure" output="Get \"https://10.132.0.53:8000/ready\": dial tcp 10.132.0.53:8000: connect: connection refused"
Jul 09 14:37:37.044673 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.044584    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:37:37.046933 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:37:37.046903    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7315-140081359760096\" is forbidden: User \"system:node:ip-10-0-143-235.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-143-235.ec2.internal' and this object" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" pod="openshift-ingress/test-curl-7315-140081359760096"
Jul 09 14:37:37.060746 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.060720    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099e61a3-6b8b-4792-b821-5017eb456a65" path="/var/lib/kubelet/pods/099e61a3-6b8b-4792-b821-5017eb456a65/volumes"
Jul 09 14:37:37.141947 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.141911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.141947 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.141950    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.142213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.141972    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.142213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142018    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.142213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142107    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k88dk\" (UniqueName: \"kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.142213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm\") pod \"09c62251-ef9a-447d-a050-0d1881f2b90c\" (UID: \"09c62251-ef9a-447d-a050-0d1881f2b90c\") "
Jul 09 14:37:37.142424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142279    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:37:37.142424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache" (OuterVolumeSpecName: "model-cache") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:37:37.142424 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142383    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home" (OuterVolumeSpecName: "home") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:37:37.142734 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142714    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-model-cache\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.142805 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142786    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-tmp-dir\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.142805 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.142798    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-home\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.144510 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.144490    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:37:37.145017 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.144996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm" (OuterVolumeSpecName: "dshm") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 14:37:37.145089 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.145020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk" (OuterVolumeSpecName: "kube-api-access-k88dk") pod "09c62251-ef9a-447d-a050-0d1881f2b90c" (UID: "09c62251-ef9a-447d-a050-0d1881f2b90c"). InnerVolumeSpecName "kube-api-access-k88dk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:37.244228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.244184    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k88dk\" (UniqueName: \"kubernetes.io/projected/09c62251-ef9a-447d-a050-0d1881f2b90c-kube-api-access-k88dk\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.244228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.244223    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09c62251-ef9a-447d-a050-0d1881f2b90c-dshm\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.244438 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.244238    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09c62251-ef9a-447d-a050-0d1881f2b90c-tls-certs\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:37.276213 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.276173    2580 generic.go:356] "Generic (PLEG): container finished" podID="194b4e14-3ebd-424e-bdd9-f6a50f133910" containerID="15c09a175a5d9409df686ff29c33010b7edb858455d58d0e57516a1ebbc563b4" exitCode=0
Jul 09 14:37:37.276398 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.276272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7315-140081359760816" event={"ID":"194b4e14-3ebd-424e-bdd9-f6a50f133910","Type":"ContainerDied","Data":"15c09a175a5d9409df686ff29c33010b7edb858455d58d0e57516a1ebbc563b4"}
Jul 09 14:37:37.277653 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.277629    2580 generic.go:356] "Generic (PLEG): container finished" podID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerID="de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9" exitCode=0
Jul 09 14:37:37.277760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.277661    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" event={"ID":"09c62251-ef9a-447d-a050-0d1881f2b90c","Type":"ContainerDied","Data":"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"}
Jul 09 14:37:37.277760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.277671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"
Jul 09 14:37:37.277760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.277688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv" event={"ID":"09c62251-ef9a-447d-a050-0d1881f2b90c","Type":"ContainerDied","Data":"8724e067d4dcd5bd25c8a25bab093009b01f7007d46f8d5f8dfebbbd1b92df38"}
Jul 09 14:37:37.277760 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.277707    2580 scope.go:117] "RemoveContainer" containerID="de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"
Jul 09 14:37:37.287102 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.287077    2580 scope.go:117] "RemoveContainer" containerID="de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"
Jul 09 14:37:37.287425 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:37:37.287401    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9\": container with ID starting with de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9 not found: ID does not exist" containerID="de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"
Jul 09 14:37:37.287473 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.287435    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9"} err="failed to get container status \"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9\": rpc error: code = NotFound desc = could not find container \"de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9\": container with ID starting with de011ff77ecf2dbf12b4e263f0acc2cab7267ae162930c7ccece0a496153d1e9 not found: ID does not exist"
Jul 09 14:37:37.306238 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.306187    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:37:37.309762 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:37.309727    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5a8685/rate-test-model-5a8685-kserve-95b84fd5f-wbnkv"]
Jul 09 14:37:38.426084 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.426054    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:38.447713 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.447688    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7315-140081359760816_194b4e14-3ebd-424e-bdd9-f6a50f133910/test-curl-7315-140081359760816/0.log"
Jul 09 14:37:38.489917 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.489880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760816"]
Jul 09 14:37:38.493815 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.493783    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7315-140081359760816"]
Jul 09 14:37:38.556616 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.556561    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7n9j\" (UniqueName: \"kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j\") pod \"194b4e14-3ebd-424e-bdd9-f6a50f133910\" (UID: \"194b4e14-3ebd-424e-bdd9-f6a50f133910\") "
Jul 09 14:37:38.558909 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.558884    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j" (OuterVolumeSpecName: "kube-api-access-m7n9j") pod "194b4e14-3ebd-424e-bdd9-f6a50f133910" (UID: "194b4e14-3ebd-424e-bdd9-f6a50f133910"). InnerVolumeSpecName "kube-api-access-m7n9j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:38.658075 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:38.658014    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7n9j\" (UniqueName: \"kubernetes.io/projected/194b4e14-3ebd-424e-bdd9-f6a50f133910-kube-api-access-m7n9j\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:39.061166 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:39.061131    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" path="/var/lib/kubelet/pods/09c62251-ef9a-447d-a050-0d1881f2b90c/volumes"
Jul 09 14:37:39.061451 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:39.061436    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="194b4e14-3ebd-424e-bdd9-f6a50f133910" path="/var/lib/kubelet/pods/194b4e14-3ebd-424e-bdd9-f6a50f133910/volumes"
Jul 09 14:37:39.289375 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:39.289351    2580 scope.go:117] "RemoveContainer" containerID="15c09a175a5d9409df686ff29c33010b7edb858455d58d0e57516a1ebbc563b4"
Jul 09 14:37:39.289375 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:39.289366    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7315-140081359760816"
Jul 09 14:37:42.493880 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.493841    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:37:42.494333 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.494157    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" podUID="b333976f-1506-4f6b-a39b-77e24d73e1e5" containerName="maas-api" containerID="cri-o://baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d" gracePeriod=30
Jul 09 14:37:42.951735 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.951702    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:37:42.998692 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.998653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnlcx\" (UniqueName: \"kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx\") pod \"b333976f-1506-4f6b-a39b-77e24d73e1e5\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") "
Jul 09 14:37:42.998897 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.998714    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") pod \"b333976f-1506-4f6b-a39b-77e24d73e1e5\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") "
Jul 09 14:37:42.998897 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.998739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca\") pod \"b333976f-1506-4f6b-a39b-77e24d73e1e5\" (UID: \"b333976f-1506-4f6b-a39b-77e24d73e1e5\") "
Jul 09 14:37:42.999191 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:42.999158    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b333976f-1506-4f6b-a39b-77e24d73e1e5" (UID: "b333976f-1506-4f6b-a39b-77e24d73e1e5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:37:43.001003 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.000978    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b333976f-1506-4f6b-a39b-77e24d73e1e5" (UID: "b333976f-1506-4f6b-a39b-77e24d73e1e5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:37:43.001093 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.000981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx" (OuterVolumeSpecName: "kube-api-access-qnlcx") pod "b333976f-1506-4f6b-a39b-77e24d73e1e5" (UID: "b333976f-1506-4f6b-a39b-77e24d73e1e5"). InnerVolumeSpecName "kube-api-access-qnlcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:43.099873 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.099828    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnlcx\" (UniqueName: \"kubernetes.io/projected/b333976f-1506-4f6b-a39b-77e24d73e1e5-kube-api-access-qnlcx\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:43.099873 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.099868    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b333976f-1506-4f6b-a39b-77e24d73e1e5-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:43.100078 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.099883    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b333976f-1506-4f6b-a39b-77e24d73e1e5-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:43.310258 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.310222    2580 generic.go:356] "Generic (PLEG): container finished" podID="b333976f-1506-4f6b-a39b-77e24d73e1e5" containerID="baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d" exitCode=0
Jul 09 14:37:43.310422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.310268    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"
Jul 09 14:37:43.310422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.310304    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" event={"ID":"b333976f-1506-4f6b-a39b-77e24d73e1e5","Type":"ContainerDied","Data":"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"}
Jul 09 14:37:43.310422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.310367    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz" event={"ID":"b333976f-1506-4f6b-a39b-77e24d73e1e5","Type":"ContainerDied","Data":"6c214969bb15ffe6eeea0bc3f7ec62ea68c40055e4e8f7bf7c3d6c253f90d9b9"}
Jul 09 14:37:43.310422 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.310397    2580 scope.go:117] "RemoveContainer" containerID="baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"
Jul 09 14:37:43.319230 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.319206    2580 scope.go:117] "RemoveContainer" containerID="baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"
Jul 09 14:37:43.319517 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:37:43.319500    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d\": container with ID starting with baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d not found: ID does not exist" containerID="baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"
Jul 09 14:37:43.319584 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.319525    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d"} err="failed to get container status \"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d\": rpc error: code = NotFound desc = could not find container \"baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d\": container with ID starting with baed52a7be051a1f3a7eba414edb78179162a2a59fbedd4b279c3fed3d792f9d not found: ID does not exist"
Jul 09 14:37:43.328171 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.328138    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:37:43.333060 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:43.333029    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-5a8685-7b56ffc69b-pwlfz"]
Jul 09 14:37:45.060439 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:45.060404    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b333976f-1506-4f6b-a39b-77e24d73e1e5" path="/var/lib/kubelet/pods/b333976f-1506-4f6b-a39b-77e24d73e1e5/volumes"
Jul 09 14:37:50.050012 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.049973    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:37:50.050479 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.050249    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" podUID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" containerName="maas-api" containerID="cri-o://1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15" gracePeriod=30
Jul 09 14:37:50.305143 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.305119    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:37:50.346853 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.346816    2580 generic.go:356] "Generic (PLEG): container finished" podID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" containerID="1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15" exitCode=0
Jul 09 14:37:50.347064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.346871    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" event={"ID":"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c","Type":"ContainerDied","Data":"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"}
Jul 09 14:37:50.347064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.346875    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"
Jul 09 14:37:50.347064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.346926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt" event={"ID":"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c","Type":"ContainerDied","Data":"40dac22fd5235b165c1b743128b34aa29c36aae73a39bd1929d312b23d7877f8"}
Jul 09 14:37:50.347064 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.346956    2580 scope.go:117] "RemoveContainer" containerID="1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"
Jul 09 14:37:50.355724 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.355706    2580 scope.go:117] "RemoveContainer" containerID="1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"
Jul 09 14:37:50.356005 ip-10-0-143-235 kubenswrapper[2580]: E0709 14:37:50.355981    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15\": container with ID starting with 1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15 not found: ID does not exist" containerID="1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"
Jul 09 14:37:50.356070 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.356017    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15"} err="failed to get container status \"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15\": rpc error: code = NotFound desc = could not find container \"1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15\": container with ID starting with 1fab74c2f564cfd2616d4aca8eb4f8d57f8b6ee84c2788e62dc6d3f299385e15 not found: ID does not exist"
Jul 09 14:37:50.370320 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.370294    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrshj\" (UniqueName: \"kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj\") pod \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") "
Jul 09 14:37:50.370470 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.370371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls\") pod \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") "
Jul 09 14:37:50.370470 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.370456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca\") pod \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\" (UID: \"8bef7f3f-d87e-42d0-9eef-dafeaf407f5c\") "
Jul 09 14:37:50.370905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.370883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" (UID: "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 14:37:50.372542 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.372524    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj" (OuterVolumeSpecName: "kube-api-access-mrshj") pod "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" (UID: "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c"). InnerVolumeSpecName "kube-api-access-mrshj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 14:37:50.372622 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.372540    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" (UID: "8bef7f3f-d87e-42d0-9eef-dafeaf407f5c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 14:37:50.472104 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.472066    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-openshift-service-ca\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:50.472104 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.472097    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrshj\" (UniqueName: \"kubernetes.io/projected/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-kube-api-access-mrshj\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:50.472104 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.472108    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c-maas-api-tls\") on node \"ip-10-0-143-235.ec2.internal\" DevicePath \"\""
Jul 09 14:37:50.669018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.668985    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:37:50.672858 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:50.672828    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-a4b0c9-7ddcc889d9-httgt"]
Jul 09 14:37:51.060488 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:51.060454    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" path="/var/lib/kubelet/pods/8bef7f3f-d87e-42d0-9eef-dafeaf407f5c/volumes"
Jul 09 14:37:57.214987 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:37:57.214946    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74c6c5b44c-p22ln_ea1c04c7-2027-417e-bb7e-de219d89e06b/authorino/0.log"
Jul 09 14:38:03.280733 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:03.280698    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-849cbdf4c4-gcn8x_f55597ad-2cad-4be9-bb96-81d28e58ef95/manager/0.log"
Jul 09 14:38:03.700270 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:03.700234    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78fdd99dcd-4trrn_2bc0e30d-568f-4b2f-831e-c9a79af58740/copy-manifests/0.log"
Jul 09 14:38:03.710354 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:03.710333    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78fdd99dcd-4trrn_2bc0e30d-568f-4b2f-831e-c9a79af58740/manager/0.log"
Jul 09 14:38:04.350970 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:04.350943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-74b776d86c-f4gnb_c0dbf1f8-de6a-4f9e-93cd-75be3d710f1d/maas-api/0.log"
Jul 09 14:38:04.478037 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:04.478006    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-9dbcv_38858016-310b-44cb-b5be-957a055a2e35/postgres/0.log"
Jul 09 14:38:05.817773 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:05.817747    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74c6c5b44c-p22ln_ea1c04c7-2027-417e-bb7e-de219d89e06b/authorino/0.log"
Jul 09 14:38:07.448511 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:07.448479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78985cb57d-4hxxk_18ed8de9-6c28-4975-8b96-7858cec7279f/kube-auth-proxy/0.log"
Jul 09 14:38:07.699468 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:07.699388    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ldp5q_f987c4fe-cd77-4961-93c2-3b0166f8b67c/payload-pre-processing/0.log"
Jul 09 14:38:08.439616 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:08.439551    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qwf6k_701d1b60-2ab5-430c-99b7-48943f74c529/main/0.log"
Jul 09 14:38:08.666018 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:08.665981    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbqbsr5_4ae68b94-ee40-4c53-97f8-2a61a208ca8e/main/0.log"
Jul 09 14:38:15.106233 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:15.106093    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:38:15.113148 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:15.111966    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:38:15.113274 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:15.113176    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:38:15.119108 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:15.119084    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:38:16.088519 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:16.088479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x5q8v_05625673-1f39-4a55-9a8a-0c55f5245758/global-pull-secret-syncer/0.log"
Jul 09 14:38:16.170606 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:16.170544    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hb59q_a03819f3-15f7-4c93-8b74-04c2cb175f88/konnectivity-agent/0.log"
Jul 09 14:38:16.294002 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:16.293972    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-235.ec2.internal_2d5108fce62f539a837eb3b8262da6a3/haproxy/0.log"
Jul 09 14:38:20.892075 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:20.891987    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-74c6c5b44c-p22ln_ea1c04c7-2027-417e-bb7e-de219d89e06b/authorino/0.log"
Jul 09 14:38:23.119284 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.119261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlgnx_f10aad93-e163-479b-a921-47331e292259/kube-state-metrics/0.log"
Jul 09 14:38:23.150675 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.150642    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlgnx_f10aad93-e163-479b-a921-47331e292259/kube-rbac-proxy-main/0.log"
Jul 09 14:38:23.177824 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.177788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlgnx_f10aad93-e163-479b-a921-47331e292259/kube-rbac-proxy-self/0.log"
Jul 09 14:38:23.381860 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.381816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdms9_73720c1c-9f25-4f26-b17a-19f5f75ed420/node-exporter/0.log"
Jul 09 14:38:23.404698 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.404668    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdms9_73720c1c-9f25-4f26-b17a-19f5f75ed420/kube-rbac-proxy/0.log"
Jul 09 14:38:23.428197 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.428166    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pdms9_73720c1c-9f25-4f26-b17a-19f5f75ed420/init-textfile/0.log"
Jul 09 14:38:23.558403 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.558374    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9xddl_2608b02b-2b5b-4bd2-a85f-3a999d3bd449/kube-rbac-proxy-main/0.log"
Jul 09 14:38:23.601818 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.601775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9xddl_2608b02b-2b5b-4bd2-a85f-3a999d3bd449/kube-rbac-proxy-self/0.log"
Jul 09 14:38:23.653832 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.653804    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9xddl_2608b02b-2b5b-4bd2-a85f-3a999d3bd449/openshift-state-metrics/0.log"
Jul 09 14:38:23.739901 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.739874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/prometheus/0.log"
Jul 09 14:38:23.778744 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.778715    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/config-reloader/0.log"
Jul 09 14:38:23.804173 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.804129    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/thanos-sidecar/0.log"
Jul 09 14:38:23.828635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.828542    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/kube-rbac-proxy-web/0.log"
Jul 09 14:38:23.852409 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.852366    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/kube-rbac-proxy/0.log"
Jul 09 14:38:23.877449 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.877417    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/kube-rbac-proxy-thanos/0.log"
Jul 09 14:38:23.900482 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.900458    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_51f5e2d2-2fb6-41fc-ab4b-06f940617333/init-config-reloader/0.log"
Jul 09 14:38:23.977313 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:23.977271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-9bp7q_43c65f41-f0bd-43e3-bd66-2a121bfece86/prometheus-operator-admission-webhook/0.log"
Jul 09 14:38:25.944290 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:25.944251    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/2.log"
Jul 09 14:38:25.950537 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:25.950501    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-qskv9_f8bca19a-836b-4f10-9d32-883deea28646/console-operator/3.log"
Jul 09 14:38:26.352221 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352184    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"]
Jul 09 14:38:26.352544 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352532    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b333976f-1506-4f6b-a39b-77e24d73e1e5" containerName="maas-api"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352547    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b333976f-1506-4f6b-a39b-77e24d73e1e5" containerName="maas-api"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352561    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerName="main"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352581    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerName="main"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352593    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" containerName="maas-api"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352599    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" containerName="maas-api"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352608    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="194b4e14-3ebd-424e-bdd9-f6a50f133910" containerName="test-curl-7315-140081359760816"
Jul 09 14:38:26.352635 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352614    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="194b4e14-3ebd-424e-bdd9-f6a50f133910" containerName="test-curl-7315-140081359760816"
Jul 09 14:38:26.352856 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352672    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="09c62251-ef9a-447d-a050-0d1881f2b90c" containerName="main"
Jul 09 14:38:26.352856 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352680    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="194b4e14-3ebd-424e-bdd9-f6a50f133910" containerName="test-curl-7315-140081359760816"
Jul 09 14:38:26.352856 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352686    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b333976f-1506-4f6b-a39b-77e24d73e1e5" containerName="maas-api"
Jul 09 14:38:26.352856 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.352692    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bef7f3f-d87e-42d0-9eef-dafeaf407f5c" containerName="maas-api"
Jul 09 14:38:26.355881 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.355859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.358341 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.358316    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4mx2l\"/\"openshift-service-ca.crt\""
Jul 09 14:38:26.358539 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.358523    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4mx2l\"/\"default-dockercfg-hwwlv\""
Jul 09 14:38:26.359289 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.359275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4mx2l\"/\"kube-root-ca.crt\""
Jul 09 14:38:26.363843 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.363821    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"]
Jul 09 14:38:26.388815 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.388784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x672m\" (UniqueName: \"kubernetes.io/projected/5ca75f31-72f0-4a42-a3cf-4da8234c5640-kube-api-access-x672m\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.388997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.388827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-lib-modules\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.388997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.388845    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-podres\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.388997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.388871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-proc\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.388997 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.388952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-sys\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489777 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489738    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-sys\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x672m\" (UniqueName: \"kubernetes.io/projected/5ca75f31-72f0-4a42-a3cf-4da8234c5640-kube-api-access-x672m\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489853    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-lib-modules\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-podres\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489873    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-sys\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.489986 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489892    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-proc\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.490173 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.489998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-proc\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.490173 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.490072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-podres\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.490173 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.490072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ca75f31-72f0-4a42-a3cf-4da8234c5640-lib-modules\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.498890 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.498862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x672m\" (UniqueName: \"kubernetes.io/projected/5ca75f31-72f0-4a42-a3cf-4da8234c5640-kube-api-access-x672m\") pod \"perf-node-gather-daemonset-9jgrk\" (UID: \"5ca75f31-72f0-4a42-a3cf-4da8234c5640\") " pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.667198 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.667084    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:26.807151 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:26.806992    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"]
Jul 09 14:38:26.809913 ip-10-0-143-235 kubenswrapper[2580]: W0709 14:38:26.809879    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ca75f31_72f0_4a42_a3cf_4da8234c5640.slice/crio-d927f0e98aa33319817e18e883318d03cdd0f0a03d68432933eb1d053393cf8b WatchSource:0}: Error finding container d927f0e98aa33319817e18e883318d03cdd0f0a03d68432933eb1d053393cf8b: Status 404 returned error can't find the container with id d927f0e98aa33319817e18e883318d03cdd0f0a03d68432933eb1d053393cf8b
Jul 09 14:38:27.533722 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:27.533686    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk" event={"ID":"5ca75f31-72f0-4a42-a3cf-4da8234c5640","Type":"ContainerStarted","Data":"f5d0b1bed318c26d7443c598c7f36e216f86fc3186147de5d318fe010490ab24"}
Jul 09 14:38:27.534157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:27.533740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk" event={"ID":"5ca75f31-72f0-4a42-a3cf-4da8234c5640","Type":"ContainerStarted","Data":"d927f0e98aa33319817e18e883318d03cdd0f0a03d68432933eb1d053393cf8b"}
Jul 09 14:38:27.534157 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:27.533830    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:27.549559 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:27.549495    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk" podStartSLOduration=1.549475597 podStartE2EDuration="1.549475597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 14:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 14:38:27.548613804 +0000 UTC m=+2413.089880607" watchObservedRunningTime="2026-07-09 14:38:27.549475597 +0000 UTC m=+2413.090742381"
Jul 09 14:38:28.298966 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:28.298936    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xlsp9_855db753-1b10-4049-a83b-e9b1b989775a/dns/0.log"
Jul 09 14:38:28.321192 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:28.321164    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xlsp9_855db753-1b10-4049-a83b-e9b1b989775a/kube-rbac-proxy/0.log"
Jul 09 14:38:28.347663 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:28.347633    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-dxqhw_42df1555-d3fb-4246-807a-e18a880a86a5/dns-node-resolver/0.log"
Jul 09 14:38:29.019203 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:29.019171    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wkbvh_57169df3-14c5-4df9-990d-77581fa9941f/node-ca/0.log"
Jul 09 14:38:30.132165 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:30.132136    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78985cb57d-4hxxk_18ed8de9-6c28-4975-8b96-7858cec7279f/kube-auth-proxy/0.log"
Jul 09 14:38:30.251905 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:30.251876    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ldp5q_f987c4fe-cd77-4961-93c2-3b0166f8b67c/payload-pre-processing/0.log"
Jul 09 14:38:31.071542 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:31.071508    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-t6xq5_1e20d41c-0a27-40b7-a77f-3b3d88e9979a/serve-healthcheck-canary/0.log"
Jul 09 14:38:31.635971 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:31.635937    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nw7vv_47cfd0c9-380d-4553-815f-ecfa50a3e591/kube-rbac-proxy/0.log"
Jul 09 14:38:31.660853 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:31.660796    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nw7vv_47cfd0c9-380d-4553-815f-ecfa50a3e591/exporter/0.log"
Jul 09 14:38:31.684311 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:31.684279    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-nw7vv_47cfd0c9-380d-4553-815f-ecfa50a3e591/extractor/0.log"
Jul 09 14:38:33.549196 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:33.549155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4mx2l/perf-node-gather-daemonset-9jgrk"
Jul 09 14:38:34.245945 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:34.245913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-849cbdf4c4-gcn8x_f55597ad-2cad-4be9-bb96-81d28e58ef95/manager/0.log"
Jul 09 14:38:34.397544 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:34.397509    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78fdd99dcd-4trrn_2bc0e30d-568f-4b2f-831e-c9a79af58740/manager/0.log"
Jul 09 14:38:34.422176 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:34.422146    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-78fdd99dcd-4trrn_2bc0e30d-568f-4b2f-831e-c9a79af58740/copy-manifests/0.log"
Jul 09 14:38:35.695578 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:35.695546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-86df669b44-hdnvf_37ada3c7-a764-4e53-8b27-bcc3c75a307b/manager/0.log"
Jul 09 14:38:35.746809 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:35.746729    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-c87q4_6393ec93-7ac6-4c28-a21b-08cb58c3f84a/openshift-lws-operator/0.log"
Jul 09 14:38:36.302383 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:36.302354    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78985cb57d-4hxxk_18ed8de9-6c28-4975-8b96-7858cec7279f/kube-auth-proxy/0.log"
Jul 09 14:38:36.395126 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:36.395091    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ldp5q_f987c4fe-cd77-4961-93c2-3b0166f8b67c/payload-pre-processing/0.log"
Jul 09 14:38:37.979139 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:37.979107    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-78985cb57d-4hxxk_18ed8de9-6c28-4975-8b96-7858cec7279f/kube-auth-proxy/0.log"
Jul 09 14:38:38.074769 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:38.074738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-ldp5q_f987c4fe-cd77-4961-93c2-3b0166f8b67c/payload-pre-processing/0.log"
Jul 09 14:38:42.800348 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.800261    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/kube-multus-additional-cni-plugins/0.log"
Jul 09 14:38:42.822512 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.822478    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/egress-router-binary-copy/0.log"
Jul 09 14:38:42.845106 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.845078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/cni-plugins/0.log"
Jul 09 14:38:42.869619 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.869553    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/bond-cni-plugin/0.log"
Jul 09 14:38:42.890742 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.890713    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/routeoverride-cni/0.log"
Jul 09 14:38:42.913298 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.913274    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/whereabouts-cni-bincopy/0.log"
Jul 09 14:38:42.936095 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.936061    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-zjd4c_1a14ee55-2bdd-4d4c-ad81-268c6a9d67a8/whereabouts-cni/0.log"
Jul 09 14:38:42.969361 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:42.969335    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lbj27_aaf4d1cb-9869-4307-93b4-7930812d4257/kube-multus/0.log"
Jul 09 14:38:43.074994 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:43.074961    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nzsrt_60bf2bd8-1417-48bb-93cd-c7e4763d6817/network-metrics-daemon/0.log"
Jul 09 14:38:43.094747 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:43.094723    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nzsrt_60bf2bd8-1417-48bb-93cd-c7e4763d6817/kube-rbac-proxy/0.log"
Jul 09 14:38:44.181228 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.181196    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-controller/0.log"
Jul 09 14:38:44.200774 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.200746    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/0.log"
Jul 09 14:38:44.200959 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.200737    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovn-acl-logging/1.log"
Jul 09 14:38:44.219218 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.219188    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/kube-rbac-proxy-node/0.log"
Jul 09 14:38:44.244033 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.243998    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 14:38:44.266996 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.266914    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/northd/0.log"
Jul 09 14:38:44.290968 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.290942    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/nbdb/0.log"
Jul 09 14:38:44.318935 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.318912    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/sbdb/0.log"
Jul 09 14:38:44.373410 ip-10-0-143-235 kubenswrapper[2580]: I0709 14:38:44.373381    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-77rxp_c0d9197b-afa8-448e-a332-0db0c62e053c/ovnkube-controller/0.log"